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Executive Summary 


The Minnesota Principals Academy (MPA) is an executive development program designed to enhance the 
performance of school leaders in order to improve educational outcomes for students. The Center for Applied 
Research and Education Improvement (CAREI) serves as the program evaluator with the intention of providing 
information on the extent to which participants, upon completion of the program, changed their behaviors or 
gained knowledge and skills in the areas covered by the MPA. This report summarizes the results from surveys 
administered across three 2017-19 MPA cohorts: Brainerd/Staples, Fergus Falls, and the Twin Cities. 


Survey Participants 

MPA participants from the three 2017-19 cohorts completed a retrospective pre/post-survey in spring 2019 
about their leadership practices, skills, and behaviors both before and after participating in the program. 
Supervisors of MPA participants also completed a survey to provide information on changes in participants in 
the areas covered by the MPA curriculum. Nearly all MPA participants completed a survey (94%, n=92/98), as 
well as the majority of supervisors (65%, n=36/55). 


Key Findings 

The survey results were overwhelmingly positive and described a pattern of change for the participants in the 
key areas identified by the MPA curriculum. One supervisor respondent stated that, “This program is a necessity 
for all principals in the state. It provides a common understanding of best educational practices, educational 
research, and a support network among administrative colleagues.” 


e MPA participants and their supervisors reported that participants gained a variety of leadership skills 
through the MPA program, emerging as stronger strategic, instructional, and transformational leaders. 


o Strategic leadership skills were reported as the area of greatest improvement for MPA participants, 
with more leaders using data and research in decision making, analyzing issues at a systems-level, 
trying new approaches in the change process at their schools, clarifying their school or district 
visions, and asking more purposeful questions as they approach problems. 


o Ways in which leaders reported improving their instructional leadership skills included having more 
conversations with teachers about instruction and how people learn, providing additional resources 
to staff to support instruction, and changing their teacher observation/evaluation process. After 
participating in the MPA, supervisors described their leaders as more student-centered (including 
prioritizing student needs when creating staff schedules) and focused on equity. 


o Changes in leaders’ transformational leadership skills related to leaders’ improved ability to work in 
a shared leadership model and gather staff perspectives, improved connections with teachers and 
staff, and leaders’ efforts to foster a culture of collaboration in schools. 


e MPA participants valued the networks they developed with principals and school leaders from other 
districts, as well as the tools and resources made available to them through their participation in the 
MPA. Supervisors concurred and discussed this as an area of impact in their schools and districts. 


e Achallenge noted by a few supervisors was that their leaders’ participation in the MPA placed a burden 


on the system and teachers, at times, due to the time required of its participants to be out of their 
buildings for the program, especially in rural or smaller districts. 
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Introduction 


Overview of the Minnesota Principals Academy 

The Minnesota Principals Academy (MPA) is an executive development program designed for school leaders that 
provides ongoing professional development to Minnesota’s school leaders by connecting theory to practice. The 
program is housed in the University of Minnesota’s Department of Organizational Leadership, Policy, and 
Development in the College of Education and Human Development, with financial support provided by the 
Minnesota Legislature and participants’ school districts.* 


The MPA’s curriculum is based on curriculum developed by the National Institute for School Leadership’s (NISL) 
Executive Development Program, which aims to provide principals with research-based information that they 
can use in their schools. The NISL curriculum includes 12 units designed to help participants in three areas that 
are key to their role: leadership skills, best practices in standards-based education, and content area instruction. 
However, University of Minnesota faculty have enhanced the content of the NISL curriculum by providing 
additional information in areas that are of interest and importance for its participants; therefore, in our report, 
references to the “MPA curriculum” refer to this modified curriculum. In addition, all participants complete an 
Action Learning Project, which is intended to encourage participants to examine an instructional issue or area 
relevant to their school. The Action Learning Project also provides a framework that the leader can use after 
program completion. 


The 2017-19 MPA participants included leaders from three cohorts: Brainerd/Staples, Fergus Falls, and the Twin 
Cities. Participants in all three cohorts began the program in fall 2017 and completed the program in spring 
2019. 


Purpose of the Report 

The Center for Applied Research and Educational Improvement (CAREI) serves as the evaluator of the MPA. The 
purpose of this evaluation is to describe and understand the MPA as a resource and training program for school 
and district leaders. Specifically, our evaluation is designed to answer the following questions: 


1. To what extent have participants gained knowledge in the areas covered by the MPA curriculum? 

2. To what extent do the supervisors of MPA participants see a change in participants’ knowledge in the 
areas covered by the MPA curriculum? 

3. To what extent have student outcomes (e.g., scores on statewide reading and math tests) in 
participants’ schools changed as a result of participation in the MPA? 


This report provides information on the first two evaluation questions for the three 2017-19 cohorts. The third 


evaluation question will be addressed in a subsequent report by analyzing and reporting on the results of the 
spring 2019 Minnesota Comprehensive Assessments (MCAs). 


1 For more information about the MPA, see https://innovation.umn.edu/mnpa/. 
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Methods 


MPA Participant Survey 

To examine the extent to which participants gained knowledge in the areas covered by the MPA curriculum 
(Evaluation Question 1), participants were asked to complete the MPA Participant Survey (see Appendix A) 
during one of their course sessions in March—April 2019. Overall, 94% of MPA participants from the 2017-19 
cohorts completed the survey (n=92/98). This included 97% of Brainerd/Staples participants (n=36/37), 91% of 
Fergus Falls participants (n=29/32), and 94% of Twin Cities participants (n=27/29). The survey was administered 
using Qualtrics, an online survey tool. Participants received an initial email inviting them to complete the survey 
and two reminder emails in the following weeks. The survey consisted of closed-ended retrospective pre/post 
survey items regarding participants’ leadership practices and their perceptions about teachers’ instructional 
practices, school-level practices, and district-level practices. The open-ended survey items focused on what 
participants perceived was the greatest “take-away” from their participation, what content they found most 
helpful, and what they are doing differently after completing the program. 


MPA Supervisor Survey 

To examine the extent to which the supervisors of MPA participants see a change in participants’ knowledge in 
the areas covered by the MPA curriculum (Evaluation Question 2), participants’ school/district supervisors were 
administered the MPA Supervisor Survey (see Appendix B) in spring 2019. Supervisors received an initial email 
inviting them to complete the online survey and two follow-up emails reminding them to complete the survey. 
In cases where multiple participants had the same supervisor, the supervisors were asked to complete only one 
survey and to think in general about all of the participants when answering survey items.”? 


Analysis and Reporting 

For closed-ended survey items, we report the number and percentage of respondents selecting each response 
option.* For the open-ended items, CAREI evaluators independently analyzed the responses and came to 
agreement on common themes that emerged; responses that fit into multiple themes were coded as such. 
Quotations are presented in italics (omitted text is represented by ellipses). In some instances, we have made 
small edits to quotations in order to improve readability but without changing the intent of the respondent. 
Finally, references to “him”/“her’” (“she”/“he”) are used interchangeably throughout the report and are not 
linked to respondent identities. 


Results 


MPA Participant Survey 

The MPA Participant Survey addressed participants’ gained knowledge in the areas covered by the MPA 
curriculum. The information provided in this section covers respondents’ current roles in their districts and data 
tables showing survey results for all three cohorts combined. Given that most items on the survey pertain to 
school-level leaders, we present results for only school-level respondents in Tables 2—9. Results for individuals in 
both school-level and district-level positions are included in Table 10 for a select number of items that pertain to 
district-level practices. Cohort-specific results are attached as Appendix C for Brainerd/Staples, Appendix D for 
Fergus Falls, and Appendix E for the Twin Cities. 


Supervisors from Staples, Fergus Falls, and Twin Cities completed surveys for 1-3 participants. And, in one case, the same 
person completed a separate survey for two participants because they were from two distinct charter schools. 

3 The Brainerd school district had 19 school-level participants in the MPA with the same supervisor. One survey across 19 
participants would not provide useful data. Thus, 12 surveys were completed by the supervisor (one for each site with MPA 
participants) which covered between 1 to 4 participants per site. 

‘Categories with a dash (-) indicate that 0% of respondents selected this option. Due to rounding, cumulative percentages 
may not add up to exactly 100%. 
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Current position. MPA participants’ survey results show that the majority of respondents held school- 
level positions (88%, n=81), which included principals, assistant principals, and other school leaders (see Table 
1). Asmall number of respondents (5%, n=5) served at the district-level (i.e., superintendent, assistant 
superintendent, district departmental directors), and 7% (n=6) reported serving in roles that were neither at the 
school- or district-level. 


Table 1. Current Position 


Brainerd/Staples Fergus Falls Twin Cities Overall 
% n n % 
School-level Positions 30 83% 26 90% 25 93% 81 88% 
panel 24 67% 21 72% 15 56% 60 65% 


Charter School Executive Director 


Assistant Principal 5 14% 3 10% 9 33% 17 19% 
Other School-Level Leaders? 1 3% 2 7% 1 4% 4 4% 
District-level Positions 4 11% - - 1 4% 5 5% 
Superintendent - - - - 1 4% 1 1% 
Assistant Superintendent 1 3% = = = = i 1% 
Department Director 3 8% - - - - 3 3% 
Other? 2 6% 3 10% 1 4% 6 7% 
Total 36 100% 29 100% 27 100% 92 100% 


“Other School-Level Leaders” includes positions listed by respondents such as Dean of Students, Director of Educational 


Programming, and Director of Athletics. 
b“Other” includes consultancy positions, state-level roles, or positions at independent education organizations. 


School-level respondents’ perceptions of their expertise. The 81 school-level respondents from the 
three cohorts were asked to rate their level of expertise in seven areas before and after their participation, using 
a scale of basic, moderate, good, or highly developed (see Table 2). Overall, the percentage of respondents who 
rated themselves as good or highly developed increased in all areas from pre- to post-program completion. The 
areas showing the greatest percentage point change in respondents selecting good or highly developed from 
pre- to post-program were generating enthusiasm for a shared vision for the school (item d; 47 percentage point 
change from 38% to 85% good/highly developed) and developing teacher leaders (item b; 46 percentage point 
change from 34% to 80% good/highly developed). The two items with the greatest change in respondents 
indicating that they are highly developed included solving problems systematically (item g; 17 percentage point 
change from 5% to 22% highly developed) and developing teacher leaders (item b; 16 percentage point change 
from 6% to 22%). Finally, the area of expertise with the greatest number of principals indicating they had 
increased their expertise beyond a basic level was motivating teachers who are reluctant to consider new 
instructional practices (item c; a 28 percentage point change from 32% to 4% basic). 


School-level respondents’ perceptions of their work. Using a scale of strongly disagree, disagree, agree, 
or strongly agree, respondents were asked to rate aspects of their instructional leadership; how they define and 
use standards; and their practices related to school vision, structures, and management (see Tables 3-5). 
Respondents were also asked to indicate the frequency with which they conducted a range of practices related 
to their interactions with teachers and use of research (see Table 6), as well as their community engagement, 
data use, and peer observations (see Table 7). 
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Table 2. Self-Reported Level of Expertise 


Pre/ Highly 

Post n |= ¥-13 (ou \V/ (oe (=) =] X-Man CL oLeLo MMe (=1V(-1 (0) oY fe 
a. Developing teamwork among teachers tee oe ae oe ae 
b. Developing teacher leaders is sh ue ae aur es 
Post 79 4% 16% 58% 22% 
c. Motivating teachers who are reluctant to consider new Pre 79 32% 44% 22% 3% 
instructional practices Post 79 4% 33% 56% 8% 
. : so Pre 79 15% 47% 33% 5% 
d. Generating enthusiasm for a shared vision for the school poet 79 ; 15% 72% 13% 
e. Analyzing and interpreting student assessment data we ie Be Bae ace ee 
Post 79 1% 23% 65% 11% 
F ‘ Ae Pre 79 9% 42% 46% 4% 
f. Using student achievement data to help make decisions poet 79 3% 13% 68% 16% 
g. Solving problems systematically by examining the whole Pre 79 18% 42% 35% 5% 
picture, rather than isolated parts Post 79 1% 19% 58% 22% 


MPA participants strongly agreed that they improved their instructional leadership skills in all categories pre- to 
post-program participation (see Table 3). The three skills with the greatest reported changes included 
encouraging and supporting innovative improvement practices (item e; 46 percentage point change from 10% to 
56% strongly agree), insisting that the way to improve student learning is through improving instruction (item b; 
45 percentage point change from 9% to 54% strongly agree), and understanding the seminal research on “how 
people learn” (item f; 40 percentage point change from 1% to 41% strongly agree). Similarly, the percentage of 
respondents selecting disagree or strongly disagree for the two items related to how people learn (items f and g) 
decreased by 76 and 58 percentage points from pre- to post-program, respectively, indicating a positive shift in 
opinions about their understanding of the seminal research on how people learn (i.e., for item f there was a 76 
percentage point change from 80% to 4% disagree/strongly disagree and, for item g, a 58 percentage point 
change from 88% to 30% disagree/strongly disagree). 


Table 3. Self-Reported Instructional Leadership Skills 


Pre/ Strongly Strongly 

Post n_ disagree Disagree Agree agree 
a. | monitor the effectiveness of instructional practice inthe Pre 80 3% 20% 74% 4% 
school. Post 80 1% 13% 59% 28% 
b. linsist that the way to improve student learning is Pre 80 3% 15% 74% 9% 
through improvement of instructional practices. Post 80 - 5% 41% 54% 
c. | challenge teachers to try innovative evidence-based Pre 80 - 34% 61% 5% 
instructional approaches. Post 80 - 8% 51% 41% 
d.! am able to motivate teachers who are reluctant to Pre 80 5% 45% 48% 3% 
consider new instructional practices. Post 80 - 11% 71% 18% 
e. | encourage and support innovative improvement Pre 79 a 13% 77% 10% 


practices such as coaching, modeling, observing practice, 


and providing feedback. Post 79 7 1% 43% 56% 
9 9 0, 9 
f. | understand the seminal research on “how people learn.” ie = ee a ne ee 
g. | ensure that my staff understands the seminal research Pre 30 24% 64% 13% z 
on how people learn to better instruct students in their 
Post 80 1% 29% 57% 13% 


classrooms. 
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Participants’ level of agreement increased from pre- to post-program in all areas pertaining to defining and using 
standards and setting expectations for teachers (see Table 4). The area with the greatest change for respondents 
who selected agree or strongly agree was to the statement, “I am comfortable defining for my teachers what 
constitutes effective standards for instructional practice” (item a; 40 percentage point change from 53% to 93% 
agree/strongly agree). This was also the item with the greatest change in respondents selecting strongly agree 


it 


(a 35 percentage point change from 5% pre-program to 40% post-program), followed by “I convey to teachers 
how important it is for classroom assessments to be carefully aligned to content standards” (item e; 30 


percentage point change from 5% to 35% strongly agree). 


Table 4. Perceptions of Defining and Using Standards 


Pre/ Strongly Strongly 

Post n disagree Disagree Agree Ey4g-{-) 
a. | am comfortable defining for my teachers what Pre 80 1% 46% 48% 5% 
constitutes effective standards for instructional practice. Post 80 - 8% 53% 40% 
b. | expect teachers to design lessons by working backward Pre 80 3% 33% 59% 6% 
from the standards and assessments. Post 80 3% 8% 59% 31% 
c. | ensure that the school has a standards-based Pre 80 5% 31% 59% 5% 
instructional system that is aligned with high performance Post 80 1% 13% 60% 26% 
standards. 
d. 1am clear in my expectation that teachers will regularly Pre 80 3% 55% 39% 4% 
meet to assess student work against the standards. Post 80 3% 24% 43% 31% 
e. | convey to teachers how important it is for classroom Pre 80 5% 40% 50% 5% 
assessments to be carefully aligned to content standards. Post 80 - 13% 53% 35% 


Respondents were next asked to self-report on eight practices related to the school vision, structures, and 
management (see Table 5). Overall, the percentage of respondents who agreed or strongly agreed that they 
conducted each area of practice increased in all areas from pre- to post-program. The statement with the 
greatest change in respondents selecting agree or strongly agree from pre- to post-program was “I discuss why 
student depth of understanding is critical to achieving our vision” (item d; 57 percentage point change from 29% 


to 86% agree/strongly agree). This was followed by “I have strategies to use in order to buffer teachers from 


distractions to their instruction” (item h; 35 percentage point change from 37% to 72% agree/strongly agree) 
and “In my school, | have institutionalized processes that ensure that the professional development system is 
aligned with a belief that all students can achieve the same high standards” (item g; 34 percentage point change 
from 39% to 73%).The item with the greatest change in respondents indicating that they strongly agree that 
they conduct the designated practice was “I make judgments about teacher performance using a variety of data 
sources, including student results” (item f; 30 percentage point change from 3% to 33% strongly agree). 
However, slightly more respondents post-program strongly disagreed that they have an effective data 
management system (item b; 5 percentage point change from 5% to 10% strongly disagree). 


Respondents were asked to indicate the frequency with which they conducted 11 practices related to teacher 
interactions (including discussions with teachers, observations, or modeling) or their use of research before and 
after their participation, using a scale of never, seldom, sometimes, or regularly (see Table 6). Overall, the 
number of respondents who conducted each practice sometimes or regularly increased in all areas from pre- to 
post-program. The most notable change in respondents selecting sometimes or regularly related to using 
research in decision-making (item k; 46 percentage point change from 48% to 94% sometimes/regularly). The 
area with the greatest change in respondents indicating that they conduct the practice regularly was in 
discussing instructional issues with teachers (item a; 44 percentage point change from 19% to 63% regularly). 
Finally, the greatest decrease in the number of principals who indicated that they never conduct the designated 


practice was in the area of modeling instructional strategies for teachers (item e; 8 percentage point change 


from 13% to 5% never). 
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Table 5. Self-Reported Practices Related to School Vision, Structures, and Management 


Strongly Strongly 
disagree Disagree Agree agree 

: Ss Pre 80 8% 46% 41% 5% 
a. | have established a clear vision for my school. Post 80 5% 31% 46% 18% 
b. | have established an effective data management system to pre 30 5% 49% 41% 5% 
collect student performance data continuously throughout 
the year. Post 80 10% 25% 53% 13% 
c. | ensure that our school's safety nets and interventions are Pre 80 3% 41% 50% 6% 
successful in getting struggling students back on track. Post 79 3% 24% 57% 16% 
d. | discuss why student depth of understanding is critical to Pre 80 13% 59% 28% 1% 
achieving our vision. Post 80 - 14% 66% 20% 
e. | search widely to benchmark our school’s performance and pra 80 10% 60% 29% 1% 
identify strategies that have enabled high-performing schools 
to sustain improvement. Post 80 1% 44% 39% 16% 
f. | make judgments about teacher performance using a Pre 80 1% 33% 64% 3% 
variety of data sources, including student results. Post 80 1% 11% 55% 33% 
g. In my school, | have institutionalized processes that ensure pre = gg A% 57% 34% 5% 
that the professional development system is aligned with a 
belief that all students can achieve the same high standards. Post 80 3% 25% 53% 20% 
h. | have strategies to use in order to buffer teachers from Pre 80 11% 53% 34% 3% 
distractions to their instruction. Post 80 3% 25% 57% 15% 


Pre/ 
How often do you... Post n Never Seldom Sometimes Regularly 
9 9 9 9 
a. Discuss instructional issues with your teachers? ee - te ae ae nse 
fe ‘0 ‘0 ‘0 
b. Discuss equity issues with your teachers? ere ie we oe ae Oye 
eet y Post 79 3% 15% 49% 33% 
me 3 : P Pre 79 3% 19% 53% 25% 
c. Visit classrooms to briefly observe instruction? Post 79 i: 6% 30% 54% 
d. Watch an entire lesson when observing classroom Pre 79 5% 15% 48% 32% 
instruction? Post 79 - 10% 42% 48% 
; : Pre 79 13% 51% 33% 4% 
.M | instruct i ? 
e. Model instructional strategies for teachers pect 79 5% 38% 13% 14% 
f. Give teachers specific ideas for how to improve their Pre 79 1% 15% 65% 19% 
instruction? Post 79 - 4% 51% 46% 
g. Ask teachers about their use of data in instructional Pre 79 5% 27% 53% 15% 
decision-making? Post 79 - 10% 38% 52% 
Pre 79 6% 33% 42% 19% 
h. Attend teach fessional | i it tings? 
end teacher professional learning community meetings -_ 79 1% 15% A1% 13% 
i. Provide or locate resources to help staff improve their Pre 79 1% 25% 53% 20% 
teaching? Post 79 - 6% 41% 53% 
j. Made judgements about teacher performance based on Pre 79 5% 24% 61% 10% 
student-level data? Post 79 1% 10% 59% 29% 
k. Use research evidence in your own decision making? Bie ie Be ae eet noe 
y a Post 79 - 6% 41% 53% 
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The area with the greatest change from pre- to post-program in responses related to community engagement, 
data use, and peer observations was for the practice of relying on systematically collected evaluation data to 
inform decision making (item b; 20 percentage point change from 10% to 30% monthly/more than once a 
month); this was driven by a change from 9% to 24% of respondents who indicated that they do this monthly 
(See Table 7). Another notable area of change in practice is in the frequency that teachers observed each other’s 
classrooms (item c); while 30% of principals indicated that this happened several times a year prior to their 
participation in the MPA, 49% indicated that this was happening several times a year post-participation, a 
change of 19 percentage points. 


Table 7. Self-Reported Frequency of Community Engagement, Data Use, and Peer Observations 
Pre/ RY=\V(-) #-] | More than 


How often do... Post n_ Annually’ times/yr Monthly once a month 
a. You communicate to the community your commitment to Pre 78 56% 36% 6% 1% 
the achievement of high performance standards by all 
students? Post 78 33% 49% 13% 5% 
b. You rely on systematically collected evaluation dataabout Pre 77 42% 48% 9% 1% 
your school in decision making? Post 78 18% 51% 24% 6% 
c. Teachers observe each other's classrooms? plea cate 206 1% ue 
i Post 78 44% 49% 4% 4% 


School-level respondents’ perceptions of their teachers’ practices. Respondents were asked to share 
their perceptions of six teacher practices before and after their own participation in the program, using a scale 
of strongly disagree, disagree, agree, or strongly agree (see Table 8). Overall, the percentage of respondents 
who strongly agreed that these practices were taking place increased in all areas from pre- to post-program 
participation. The two areas with the greatest change in respondents selecting agree or strongly agree from pre- 


to post-program included “Teachers in this school have ongoing conversations among themselves about 
instructional practices” (item d; 29 percentage point change from 50% to 79% agree/strongly agree) and 


y 


“Teachers in this school view problems as issues to be solved, not as barriers to action” (item f; 23 percentage 
point change from 37% to 60%). The two items with the greatest change in respondents indicating that they 
strongly agreed the practices were taking place included teachers having a collective responsibility for student 
learning (item b; 16 percentage point change from 4% to 20% strongly agree) and, as above, teachers having 
ongoing conversations about instructional practices (item d; 14 percentage point change from 5% to 19% 
strongly agree). The number of principals who strongly disagreed that their teachers viewed problems as issues 
to be solved rather than barriers to action decreased from 14% to 4% (item f; a 10 percentage point change). 


Table 8. Perceptions of Teacher Practices 


Pre/ Strongly Strongly 
Teachers in this school... Post n_ disagree Disagree Agree Eyig-{-] 
a. Are able to accurately monitor the progress of their Pre 80 3% 31% 56% 10% 
students. Post 80 4% 23% 57% 16% 
b. Have a sense of collective responsibility for student learning. ig ste “ aoe a aa 
Post 80 1% 23% 56% 20% 
c. Have the knowledge and skills they need to improve student Pre 80 1% 34% 60% 5% 
learning. Post 80 = 25% 63% 13% 
d. Have ongoing conversations among themselves about Pre 80 9% 41% 45% 5% 
instructional practices. Post 80 3% 19% 60% 19% 
e. Prepare all students to go on to some sort of post-secondary Pre 79 4% 33% 59% 4% 
education. Post 79 1% 32% 58% 9% 
f. View problems as issues to be solved, not as barriers to Pre 79 14% 49% 34% 3% 
action. Post 80 4% 36% 46% 14% 
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School-level respondents’ perceptions of their schools. Respondents were asked to indicate their 
perceptions of seven school-level practices before and after their own participation in the program; an 
additional item related to parental expectations for students to attend post-secondary education (see Table 9). 
Overall, the percentage of respondents who strongly agreed that these seven school-level practices were taking 
place increased in all areas from pre- to post-program participation. The area with the greatest change in 
respondents selecting agree or strongly agree from pre- to post-program was “Team leadership at all levels 
focuses on the quidance, direction, and support of sustained improvement in instructional practice and student 
learning” (item g; 27 percentage point change from 38% to 65% agree/strongly agree). The two items with the 
greatest change in respondents indicating that they strongly agreed that these practices were taking place 
include “Administrators and teachers collectively plan who will provide leadership for initiatives” (item d; 17 
percentage point change from 8% to 25% strongly agree) and “Our school improvement plan drives teachers’ 
professional development” (item c; 15 percentage point change from 8% to 23% strongly agree). There was a 
small but positive 7 percentage point change from pre- to post-participation in the number of respondents who 


strongly agreed that most parents in their school “expect their children to go on to some sort of post-secondary 


education” (item h; from 11% to 18%). 


Table 9. Perceptions of School-Level Practices 


Pre/ Strongly Strongly 
Post n disagree Disagree Agree agree 
a. We provide an aligned curriculum for students across the Pre 80 4% 39% 50% 8% 
grades. Post 80 6% 21% 54% 19% 
b. There is a strong commitment in this school to a common set Pre 80 3% 49% 39% 10% 
of shared goals. Post 80 4% 31% 46% 19% 
c. Our school improvement plan drives teachers’ professional Pre 80 4% 46% 43% 8% 
development. Post 80 3% 24% 51% 23% 
d. Administrators and teachers collectively plan who will provide Pre 80 8% 38% 48% 8% 
leadership for initiatives. Post 80 3% 19% 54% 25% 
e. The schedule allows for adequate embedded time for Pre 80 11% 30% 44% 15% 
collaborative teacher planning. Post 80 9% 20% 51% 20% 
f. We have a range of differentiated safety nets available at the Pre 80 8% 50% 40% 3% 
individual student level. Post 80 6% 34% 51% 9% 
g. Team leadership at all levels focuses on the guidance, Pre 80 6% 56% 33% 5% 
direction, and support of sustained improvement in Post 80 : 35% 50% 15% 
instructional practice and student learning. 
h. Most of the parents of students in this school expect their Pre 80 5% 23% 61% 11% 
children to go on to some sort of post-secondary education. Post 80 1% 20% 61% 18% 


Respondents’ perceptions of their districts. The last set of items asked respondents to consider 11 
items pertaining to district-level practices before and after their own participation in the program, using a scale 
of strongly disagree, disagree, agree, or strongly agree (see Table 10). Recall that the results for these items 
include respondents in both school- and district-level positions (n=86). These items represented the greatest 
variability in responses compared to prior items asked on the survey, with 7 items increasing in the number of 
respondents agreeing or strongly agreeing that the practice was taking place, 1 item remaining unchanged, and 
3 items decreasing in the number of respondents agreeing or strongly agreeing that the practice was taking 
place in their districts. 


The area with the greatest change in respondents selecting agree or strongly agree from pre- to post-program 


was “Principals participate in ongoing high-quality professional development” (item i; 34 percentage point 


change from 42% to 76% agree/strongly agree). This change was driven by a large shift in respondents selecting 
strongly agree post-program, changing from 6% to 38%. The next item with the greatest change in respondents 
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indicating that they strongly agreed practices were taking place was “Instructional leadership is a key component 
of our principal evaluation system” (item b; 29 percentage point change from 11% to 40% strongly agree). 


Table 10. Perceptions of District-Level Practices 


Pre/ Strongly Strongly 
Post n disagree Disagree Agree agree 
a. District staff help to ensure that our schools offer an aligned Pre 85 = 25% 65% 11% 
curriculum. Post 85 = 14% 67% 19% 
b. Instructional leadership is a key component of our principal Pre 84 - 23% 67% 11% 
evaluation system. Post 83 4% 23% 34% 40% 
c. District leaders often discuss instructional issues with school Pre 84 4% 50% 42% 5% 
leadership teams. Post 84 10% 31% 43% 17% 
d. Our district offers a range of differentiated safety nets or Pre 85 9% 34% 47% 9% 
interventions available at the individual student level. Post 85 11% 32% 48% 9% 
e. Our district has strategies to support principals and teachers pre 35 1% 25% 68% 6% 
in preparing students for college or other post-secondary 
education beyond high school. Post = 85 2% 27% 64% 7% 
f. District administrators model high levels of professional Pre 85 - 13% 76% 11% 
practice. Post 85 4% 13% 54% 29% 
g. Our district culture is one in which all teachers and Pre 35 6% 338% A7% 9% 
administrators feel responsible for working together to 
Post 85 4% 35% 38% 24% 


improve student achievement. 
h. Our district has institutionalized processes that ensure the Pre 85 A% A7% A7% 2% 
professional development program is aligned with a belief 


that all students can achieve the same high standards. Post 81 6% 46% 40% 9% 

i. Principals participate in ongoing high-quality professional Pre 85 7% 51% 36% 6% 
development. Post 85 9% 15% 38% 38% 

j. Teaching is aligned to rigorous performance standards. ee ad aes sais ote os 
Post 83 7% 28% 51% 14% 

k. Our district supports equity-related initiatives. Ble ge we eee g00 dhs 
Post 85 11% 28% 52% 9% 


Respondents’ feedback about their experience. Participants were asked to respond to the following 
open-ended questions regarding their thoughts about the MPA, each of which we review further below: 


1. What is your greatest “take-away” from your participation in the MPA? (Please name /discuss only one.) 

What content (unit or otherwise) did you find to be the most helpful to you, and why? 

3. What are you doing differently now, if anything, in your school as a result of this experience? How do 
you see it impacting teachers and/or students? 


y 


Comments from all survey respondents (school-level, district-level, and those in other positions) were included 
in our analysis, which we categorized into three broad leadership themes for reporting purposes. These 
practices related to strategic leadership, instructional leadership, and transformational leadership, which we 
define as follows: 


Strategic leadership “is based on long-term planning. It involves establishing and maintaining systems, 
allocating resources, and communicating vision. Principals need to maintain the focus clearly on the 


central vision for their school” (New Zealand Ministry of Education, 2019).° 


5 See: http://www.educationalleaders.govt.nz/Leading-change/Strategic-leadership 
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Instructional leadership “involves setting clear goals, managing curriculum, monitoring lesson plans, 
allocating resources and evaluating teachers regularly to promote student learning and growth. Quality 
of instruction is the top priority for the instructional principal. Instructional leadership is committed to 
the core business of teaching, learning, and knowledge” (Concordia University-Portland, 2013).° 


Transformational leadership “in schools is when a leader empowers members of the learning 
community to improve from within. The transformational leader does not simply run a school, merely 
keeping it afloat. Instead, leaders seek to make things better through genuine collaboration between 
the school’s members and stakeholders” (Concordia University-Portland, 2018).’ 


Respondents’ greatest take-away. First, respondents were asked to share their greatest take-away from 
the program. Of the respondents who answered this item (n=88), the five primary themes that emerged related 
to strategic leadership practices (n=47), instructional leadership practices (n=33), networking and resources 
available through their experience with the MPA (n=11), transformational leadership practices (n=5), and a 
greater understanding of their role as a change agent in their school or district (n=4). Additionally, one 
respondent wrote “everything” and one respondent provided a suggestion for the program. As a reminder, 
feedback may have been coded into multiple themes if the respondent discussed multiple take-aways. 


Strategic leadership practices. First, the majority of respondents (53%, n=47) mentioned an aspect of 
strategic leadership as the greatest take-away of their participation in the MPA, with a focus on a greater 
understanding of the importance of using data and research in thinking and planning (n=32). For example, 
several respondents wrote the following: 


The need to look at data and research when it comes to making decisions as opposed to "gut 
level" or what | "think" is the right approach to a decision or problem. 


My thinking about the change process has transformed and will forever be steeped in research- 
based evidence and has allowed me to already transform who |! amas a principal. 


My greatest take-away was the power of using research and data in supporting discussions 
surrounding any new initiatives, decisions, interventions and supports, etc. While | agree that 
this should have been a no-brainer, | do not feel like we did this well as a District and it's 
something that | hope to continue to use as | advocate for my building, in particular. 


Using research to support decision-making, to support the teacher learner. It's not about 
grabbing the next initiative to make change but digging in and understanding what the research 
says to support a change and make change happen. 


Respondents who mentioned aspects of strategic leadership commented on gaining a new perspective on the 
importance of keeping the “big picture” in mind (n=12) and their improved ability to make strategic decisions 
based on research and data (n=9). As a few of these respondents shared: 


The need to evaluate our overall system for the hidden gaps. This could be curricular with under- 
representation, achievement gap and lack of reduction, missed opportunities, etc. The greatest 
"take-away" is becoming a much-improved system-level thinker. 


8 See: https://education.cu-portland.edu/blog/leaders-link/four-instructional-leadership-skills-principals-need/ 
7 See: https://education.cu-portland.edu/blog/leaders-link/transformational-leadership-model/ 
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| need to keep my eye on overall improvement and not get stuck in day to day operations of the 
school. 


Learning to analyze initiatives/problems with more deliberation and using more research to 
support/guide decisions. | have learned to be ok with taking more time to contemplate issues 
and to ask for more help from colleagues. 


Instructional leadership practices. Second, areas of instructional leadership were mentioned as the 
greatest take-away by 38% of respondents (n=33). Of the participants who commented on this area, over 80% 
(n=27) mentioned the MPA’s content related to “how people learn” as the greatest take-away, while 18% of 
respondents (n=6) mentioned a new focus on instructional leadership or deeper knowledge in content areas 
such as math or science. Some of these participants shared the following: 


The way people learn has been the central theme that has surfaced during all units and is the 
measure against anything | have attempted at the building level. 


The book "How People Learn" has definitely impacted my own viewpoints on learning and has 
been foundational for me for the rest of the work we have done in the MPA. 


Creating an instructional practice plan with my leadership team. This has created a road map for 
us to follow and a systematic process for implementation of new initiatives. 


Networking and resources. The third area that emerged as a take-away for respondents was related to 
participants’ expanded leadership network and access to resources that were made available through their 
participation (13%, n=11). These respondents focused on connecting with principals and school leaders in other 
districts (n=7) and or the tools or resources provided by MPA facilitators that they can now use in their schools 
or districts (n=2). These respondents described their greatest take-aways as follows: 


The level of intense and deep professional learning and collaboration with colleagues. 


| learned a lot but | also learned the importance of connecting and networking with my fellow 
Principals in a meaningful way. Taking time out to connect and talk about student learning was 
SO INCREDIBLY VALUABLE! We don't often make time to do that. 


Having the tools and the research to begin creating a shared mission, vision and strategic plan for 
our District. 


Transformational leadership practices. Fourth, 6% of respondents (n=5) wrote that their greatest take- 
away was their improved transformational leadership skills in that they felt better able to work in a shared 
leadership model and gather staff perspectives. One person stated, “/ have realized that for my school to make 
more positive growth in any arena, | need to work with colleagues more and involve them more in the work to 
institutionalize and ensure implementation with fidelity.” 


New understanding of administrative role. A greater understanding of how the administrator can act as 
a change agent was considered the greatest take-away for 5% of survey respondents (n=4). One person 
described this by sharing, “/ have a better sense of how important this position is and truly how much power the 
principal has to shape so much of what is happening in their building.” 
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Suggestion. One respondent offered a suggestion related to the MPA presenters and facilitators. This 


respondent shared the following: 


area. W 


Responses varied and spanned nearly all units. It is important to note that participants did not always explicitly 


The greatest take-away is learning from other peers that have vast experiences in education, as 
principals! Too often we hear from people who are rooted in research and have “read a lot” about 
equity, pedagogy, data, instruction, etc., but they have little to no experience as a principal. MPA 
needs facilitators and leaders who have years of experience, not just as researchers, but as building 
leaders who have served in diverse schools and experienced difficult times. These are the people 
MPA should seek out. 


Most helpful content or unit for participants. For the second open-ended question, participants were 
asked what content (unit or otherwise) they found to be the most helpful, and why. Overall, 85 respondents 
provided feedback on this item; some identified multiple units that were helpful and some indicated a single 


e present the number of respondents that described each MPA unit as most helpful in Table 11. 


state which unit was most helpful, but rather they described particular topics or content that they found useful. 


In these instances, evaluators consulted course syllabi to identify the appropriate unit as best as possible, or 


consulted with the MPA Director. 


Table 11. MPA Units Participants Found Most Helpful 


Unit n % 

Unit 1: The Educational Challenge - - 

Unit 2: The Principal as Strategic Thinker 11 13% 
Unit 3: Elements of Standards-aligned Instructional Systems 3 4% 
Unit 4: Foundations of Effective Learning 22 26% 
Unit 5: Leadership in the Instructional Core Part 1: Language Arts and History 8 10% 
Unit 6: Leadership in the Instructional Core Part 2: Math and Science 10 12% 
Unit 7: Coaching for High Quality Teaching 2 2% 
Unit 8: Promoting the Learning Organization 4 5% 
Unit 9: Teams for Instructional Leadership 7 8% 
Unit 10: Ethical Leadership for Equity 16 19% 
Unit 11: Driving and Sustaining Transformation 6 7% 
Unit 12: Action Learning Project 1 1% 


As we display in Table 11, approximately one-quarter of respondents (26%, n=22) indicated that Unit 4: 
Foundations of Effective Learning was the most helpful unit, while 19% of respondents (n=16) described content 
learned as part of Unit 10: Ethical Leadership for Equity as most helpful. Some of the feedback from respondents 


about t 


hese two units included: 


When we discuss how people learn, one the biggest components for me is helping my staff to 
evaluate the prior knowledge of our students and their cultural biases or background rather than 
make assumptions that all students come with the same knowledge base. Context has become 
increasingly important in our discussions. 


| keep coming back to How People Learn because it has challenged my thinking about how | work 
with adults as learners. 


| really found the work on how people learn to be helpful. It's provided a good focus for staff 
evaluations and a lot of the conversations I've had with staff about improvement. 
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Foundations for Effective Learning. Learning about how people learn and coming back to how 
people learn in just about every section. | kept my notecard near at all times and will be moving 
forward. 


Chapter 10 — Equity. That chapter opened my eyes about unconscious white bias. | went back to 
my building that Monday and changed some practices. | look at leadership through a new lens. 


The unit on equity. It made me reflect on all the other units and | still am. | am now scanning 
practices and looking for sign-posts of inequity. It has forced me to reflect on all we do. 


The unit on equity work broadened my view on the need and urgency to prioritize this work. The 
activity where we put moments in history around the room was so powerful and it was obvious 
that in education, we have been doing the same thing for over 200 years - and it needs to stop. 


Although the majority of respondents identified a particular MPA unit or unit content areas as what was most 
helpful for them, 9 participants (11%) indicated the information on Getting Relationships Right (a presentation 
by the Search Institute) was most helpful. For example: 


The unit on relationships because | believe it's the most important thing we do AND it centers 
around my ALP [Action Learning Plan] project on adding an Advisory Schedule to our school. 


Getting relationships right and driving and sustaining transformation were the most impactful 
units for me. | think that our school has always done a great job of building relationships and 
prided itself on that, but | feel that there are some things that we could definitely change that 
would impact things ona greater level. 


Finally, 7 respondents (8%) reported that all the content covered was equally helpful, and 4 respondents 
(5%) mentioned that they found that making connections with other principals and the facilitators was 
the most helpful for them. As some participants shared: 


There were many units that really shook me to my core and caused me to examine my life as a 
principal. 


| don't think | could pick one [unit]. | can truly say that after 19 years as a principal, this has been 
the best PD | could have ever imagined. This has changed my leadership vision, and I'm confident 
| now have the tools to better lead my teachers and school. 


What MPA participants are doing differently now. MPA participants were asked what, if anything, they 
are doing differently in their school as a result of participating in the program. Of the 86 responses to this item, 
five themes emerged: changes in strategic leadership (n=45); changes in instructional leadership (n=29); changes 
in transformational leadership (n=21); personal changes such as increased confidence (n=10); and no changes at 
the time of the survey (n=5). As a reminder, feedback was coded into multiple themes if the respondent 
discussed multiple things they were doing differently. 


Changes in strategic leadership practices. Approximately half of respondents (52%, n=45) mentioned 
they were doing something related to strategic leadership differently now based on what they had learned in 
the MPA. The greatest change in this area related to making decisions more strategically (n=28). Other changes 
related to using a new approach to the change process with their schools (n=12), clarifying their school or 
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district vision (n=11), looking at issues from a systems-level (n=8), and asking more questions and being 
purposeful about asking “why” (n=6). Feedback from some of these participants about their changes include: 


My decisions around professional development and resource allocation are more data based and 
strategically aligned to the vision. 


I'm more strategic and intentional with changes and initiatives. | feel like I'm actually moving 
things a bit slower with the idea that over the long-term practices and results will be sustained 
and embedded in our culture. 


| have used the Conceptual Framework for Strategic Thinking when problem solving, especially the 
strategy steps. When | have taken the time to be a strategic thinker, | am more thoughtful, 
organized and can articulate the decision- making process. 


Prior to doing things, | am very purposeful in asking the "why" and making sure those who are a 
part of the specific process are clear on the "why." 


| focus more on the whole picture when assessing our needs and...share what | am seeing for input 
from staff before jumping into solutions. 


Changes in instructional leadership practices. Approximately one-third of participants (34%, n=29) 
indicated they are now doing something differently related to their instructional leadership. Of these 
respondents, 69% (n=20) reported having more conversations with teachers about instruction and how people 
learn. Others mentioned providing additional resources to staff (n=7) and changing the teacher 
observation/evaluation process (n=5). For example: 


Having stronger conversations with staff regarding How People Learn and how to improve 
instructional practice. 


When discussing instructional practices or observing classroom teachers, my conversations now 
include asking questions about how teachers are accessing preconceptions, creating conceptual 
frameworks, and incorporating opportunities to demonstrate metacognition. 


Aligned instructional systems and practices and promoting an organization of learning with a 
focus on equity are areas where my work will look significantly different. Teachers need to 
understand what good teaching and student engagement looks like in multiple contexts...It's the 
responsibilities of administrators to provide guidance and support around that work while 
maintaining high and supportive expectations for students and teachers. 


! am working hard on tying teacher performance to student performance, and creating an 
evaluation system that involves teacher peers observing each other. 


Using research in my feedback, being ok with slowing down and showing up in conversations 
around decisions much differently. I'm much more thoughtful prior to decision-making. | think by 
modeling the use of research and sharing it, teachers are more likely to do the same which will 
directly impact our students. 
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Changes in transformational leadership practices. Next, changes in transformational leadership practices 
were mentioned by 24% of respondents (n=21). The changes reported in this category related to creating more 
distributed leadership and developing leaders within their schools (n=13) and improving relationships and 
engagement with staff and students (n=8). As some of these respondents shared: 


Using teacher leaders to drive the instructional changes knowing they have done their research 
and are focused on our vision/mission. 


|! am working on delegating some of those tasks to allow for more opportunity to be focused on 
instruction. | have been talking more with peers and identifying some leaders that might be a part 
of a process moving forward as a site leadership team. 


Purposefully building shared leadership and modeling during staff meetings and PD. 


Personal changes. Some respondents (n=10) felt they had made changes on a more personal level as a 
result of their participation in the MPA, such as noticing changes in their confidence level, their ability to lead, 
and their use of time or resources. Respondents shared the following comments: 


| feel more confident in my own knowledge in regards to educational best practices, and | feel that 
| have a place to start from when discussing opportunity and change with my team and with the 
district team. 


| am better able to take context into consideration after seeing the different perspectives from 
participants as well as the wide range of presenters and readings. | feel much more well-rounded 
instead of relying only on my narrow set of strengths that | had before the MPA. "If all you have is 
a hammer, everything looks like a nail." Now | have more tools and a deeper understanding of 
how/when to use them. 


!am more intentional about my daily work. 
| felt | gained a wealth of knowledge and learned to utilize my resources much better. 


No changes. Five respondents (n=5) indicated that they had not, as of yet, made any specific changes. 
However, several of these respondents discussed needing more time to process what they had learned in the 
MPA and then they would be able to create actionable next steps. For example, one respondent commented, “/ 
am not doing anything differently at present....1 need to take some time to review the content and experiences of 
this program in order to determine my next steps regarding actions.” 


MPA Supervisor Survey 

This section describes the results of the MPA Supervisor Survey, which was designed to address Evaluation 
Question 2: “To what extent do the supervisors of MPA participants see a change in participants’ knowledge in 
the areas covered by the MPA curriculum?” 


Response rates. Of the 55 supervisors identified by MPA participants from the 2017-19 cohorts, 36 
completed a survey between April-May 2019, for a response rate of 65%. This included a response rate of 90% 
for Brainerd/Staples (n=9/10), a response rate of 55% for Fergus Falls (n= 11/20), and a response rate of 64% for 
the Twin Cities (n= 16/25). The 36 supervisors represented 71% of MPA participants (n=70/98), including 53% of 
participants from Fergus Falls, 95% from Brainerd/Staples, and 62% from the Twin Cities. 
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Supervisors of participants from Staples (we discuss Brainerd separately, below), Fergus Falls, and the Twin 
Cities were asked to complete only one survey, even if they supervised more than one participant; in cases 
where they supervised more than one participant, they were asked to think in general about all of the 
participants when answering the survey items. Meanwhile, the MPA participants from Brainerd had a single 
supervisor who completed the supervisor surveys. In order to provide more meaningful feedback about the 
varied MPA participant experiences and impacts, this supervisor completed separate surveys for smaller groups 
of MPA participants. In Table 12, we present key points regarding survey administration across the cohorts/sites, 
which resulted in 48 survey completions. 


Table 12. Overview of MPA Supervisor Survey Administration, by Site 


Cohort / 

District Number of surveys completed, with explanation Number of participants reviewed 

Staples, Total survey completions = 36 e On average, supervisors for the Twin Cities 

Fergus Falls, | ¢ Nearly all supervisors (n=34/35) completed 1 survey. cohort completed a survey about 1 

Twin Cities | e One supervisor completed 2 surveys — a separate survey participant, though this ranged from 1-2. 
for participants from 2 distinct charter schools. e On average, Staples and Fergus Falls 


supervisors completed a survey about 2 
participants, though this ranged from 1-3. 


Brainerd Total survey completions = 12 e On average, the Brainerd supervisor 
e One supervisor completed a separate survey for 12 completed a survey about 2 participants 
distinct sites. per site, though this ranged from 1-4. 


Supervisors’ perceptions of participants’ skills and abilities after participating in MPA. Respondents 
were asked to indicate the extent to which they agreed that participants’ improved their leadership skills as a 
result of participation in the MPA. The majority of respondents (94%) agreed or strongly agreed that leadership 
skills have improved while 6% disagreed (see Figure 1). No respondents strongly disagreed with this item. 


inl aa 
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@ Strongly disagree Disagree Agree Strongly agree 


Figure 1. Level of Agreement that Leadership Skills Improved 


Supervisors’ level of agreement with statements pertaining to improvements in MPA participants’ skill levels in 
instructional leadership, transformational leadership, and strategic leadership are discussed in the following 
sections and shown in Tables 13-15 below. 


The supervisors responded to four statements about instructional leadership skills using a 4-point scale from 
strongly disagree to strongly agree (see Table 13). More than 90% of the respondents agreed or strongly agreed 
with each statement, and no respondents strongly disagreed with any statement. Over 40% of the respondents 


strongly agreed that the participants improved in three areas: in their “commitment to effort-based theories of 


y 


learning where all students can learn” (item a), in their “ability to ensure that teachers use data to make 
instructional decisions for students” (item b), and in their “ability to identify research and best practices that 


support instructional improvement” (item d). Fewer supervisors strongly agreed that participants have improved 
their ability to regularly monitor and analyze a range of student performance data to make judgments about 
areas of curriculum and instruction that need attention (item c; 24% strongly agree). 
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Table 13. Supervisors’ Perceptions of Changes in Participants’ Instructional Leadership Skills 
Strongly Strongly 


n disagree Disagree Agree agree 
a. The participant(s) has improved their commitment to effort-based 


. : 47 - 49 539 439 
theories of learning where all students can learn. ‘ ae ‘ 
b. The participant(s) Nas Improved Selr ability to ensure that teachers 48 7 A% 54% 42% 
use data to make instructional decisions for students. 
c. The participant(s) has improved their ability to regularly monitor 
and analyze a range of student performance data to make AG ; 1% 70% 24% 
judgments about areas of curriculum and instruction that need 
attention. 
d. The participant(s) has improved their ability to identify research and 
Pelelean yl nee Heese gine. 47 2 6% 49% 45% 


best practices that support instructional improvement. 


Supervisors were asked to respond to a series of five statements related to participants’ transformational 
leadership skills (see Table 14). Supervisors’ level of agreement was high with 88% or more of the respondents 
selecting either agree or strongly agree for all statements. In particular, nearly all supervisors (96%) either 
agreed or strongly agreed that their participants have improved their team-building skills (item a). No 
respondent strongly disagreed with any of the items. 


Table 14. Supervisors’ Perceptions of Changes in Participants’ Transformational Leadership Skills 
Strongly Strongly 
n_ disagree Disagree Agree agree 
a. The participant(s) has improved their team-building skills (e.g., they 
promote professional learning with staff members, share leadership, 48 - 4% 69% 27% 
support ideas offered by team members). 
b. The participant(s) has improved their ability to motivate teachers or 
other staff members who are reluctant to consider new instructional 48 - 13% 63% 25% 
practices. 
c. The participant(s) has improved their ability to recognize the 
individual needs of their staff members and to support them 47 - 6% 57% 36% 
accordingly. 
d. The participant(s) has improved their ability to create a compelling 
school vision in which all students are held to high expectations. 
e. The participant(s) has improved their ability to listen to community 
members, advocates, or other stakeholders to inform change within 46 - 9% 61% 30% 
the school. 


47 9% 51% 40% 


Finally, supervisors were asked to consider two statements related to strategic leadership skills. The majority of 
respondents agreed or strongly agreed that participants’ skills had improved in this area as a result of 
participating in the MPA. The majority of respondents (94%) agreed or strongly agreed that participants had 
“improved their ability to think more systematically about how to improve instruction” (item a), and 89% agreed 
or strongly agreed that participants had “improved their ability to ensure that adequate resources are prioritized 


and allocated to the initiatives that will achieve maximum benefit in improving learning for all students” (item b). 
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Table 15. Supervisors’ Perceptions of Changes in Participants’ Strategic Leadership Skills 


Strongly Strongly 
n disagree Disagree Agree agree 
a. The participant(s) has improved their ability to think more 48 ; 6% 0% 54% 


systemically about how to improve instruction. 

b. The participant(s) has improved their ability to ensure that adequate 
resources are prioritized and allocated to the initiatives that will 47 - 11% 68% 21% 
achieve maximum benefit in improving learning for all students. 


Perceived impact of participants’ participation on the district. Supervisors were asked to share how 
participation in the MPA by one or more of their leaders (MPA participants) has impacted either their district or 
specific schools in their district. Of the 48 surveys, 38 included feedback, with the majority (95%, n=36) 
describing ways in which their district or schools have benefitted. The impacts of the MPA that supervisors 
described included changes in leaders’ strategic leadership practices (n=21), changes in instructional leadership 
practices (n=19), changes in transformational leadership practices (n=14), participants’ expanded leadership 
network (n=14), and other impacts (n=3). In addition, 5 supervisors described how their MPA leaders’ 
participation in the program either presented some challenges for the district (n=4) or leaders did not make any 
significant changes as a result of their participation (n=1). The areas of greatest impact that supervisors’ 
described regarding the MPA are provided in the sections below. 


Changes in strategic leadership practices. The majority of supervisors (55%, n=21/38) mentioned an 
aspect of strategic leadership as the area of greatest impact as a result of their leader(s) participating in the 
MPA. Many supervisors explained that their leaders are better able to create strategic plans with the “big 
picture” in mind, create missions and visions for their schools, and take a more systematic approach to student 
learning (n=13). Some supervisors shared the following: 


The participation of our Assistant Principal has resulted in a leader who is more strategic, 
focused, data based, and visionary in her leadership. Her participation has resulted in improved 
systems, structures, teacher coaching, and outcomes for our students. The improvement in her 
leadership has been dramatic as a result of participating in the MPA. 


[Name] has made significant growth through his time in the MPA. He actively took the learning 
on, and it has changed how he develops, implements, and leads change in his building. 


[Name] was assigned as the Principal at the beginning of this past school year. She was able to 

start the year with staff and her leadership team to create a clear mission, vision, and set 

strategic direction for the school. [Name]'s staff have stated that they feel there are higher 

standards and that they are clearly articulated to them. 

It helped him to establish the vision and strategic direction with his leadership team. 
Supervisors who mentioned aspects of strategic leadership also commented on their leaders’ improved ability to 
use or understand research and data (n=13) or that the leaders have shared research or other information they 
learned from the MPA with their staff (n=5). As a few of these respondents shared: 


He is more thoughtful and research-based in his decision making. 


She stretched herself to read more research and to grow in her understanding of data and 
continuous improvement for a district, verses a grade level. 
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As she has presented to the school board, she has utilized research, systematic approaches and 
research, to develop compelling arguments of why technology is important for our curriculum 
and student learning. 


She is developing her skills to systematically review data to improve student outcomes. Oh - and 
her 4 year graduation rate doubled this past year... Yes - she took this learning and GREW for our 
kids. 


[Name]'s biggest area of growth from the MPA was to find research and evidence based 
practices to create a plan to initiate a change within a system. She is able to look more 
systematically at her data and bring her staff back to the "why" things are needed and 
imperative through student data. 


[Name] was able to connect practices to research which helped staff see the need to change. 
Readings around equity were shared with the building leadership team which provided an on- 
ramp for the team to talk about racial bias and equity. There were pieces of research around 
assessment and instruction which resulted in looking at different criteria as we identify students 
for support courses. 


Changes in instructional leadership practices. |mpacts in the area of instructional leadership were 
mentioned by 50% of supervisors (n=19). Approximately half of these supervisors (n=9) described how MPA 
participants improved their work with teacher evaluations, coaching, or observations. For example: 

[Name] also expressed that she is having more meaningful conversations with her staff during 

their evaluations around student data and that she is developing her skills to systematically 

review data to improve student outcomes. 

Her coaching of teachers has significantly strengthened. 

Other supervisors described changes in instructional leadership practices, including how participants were now 
more student-centered (including prioritizing student needs when creating staff schedules) and focused on 
equity (n=3); more confident in their curricular decisions (n=1); or bringing new ideas to the district (n=1). As 


some of these supervisors explained: 


Two of the principals that participated in MPA have used the knowledge gained in their work to 
significantly change their staffing schedules in order to better meet the needs of all students. 


The principal attending the MPA has become more equity focused in his work. He has made it a 
priority in his school and in his personal professional growth. 


She is motivated to ensure that all of her decisions are student-centered and not adult-centered. 
She appears more confident in her curricular decisions that she makes. 


Our principal has brought very interesting and innovative ideas she's been exposed to back to the 
district. One example is implementing standards-based report cards in the elementary. 
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Changes in transformational leadership practices. A third area of impact described by 37% of 
supervisors (n=14) related to changes in transformational leadership practices as a result of the school leaders 
participating in the MPA. Half of these respondents (n=7) explained that the participants they supervise have 
improved in their ability to connect or engage with their teachers and staff. This is being accomplished through 
improved relationships, a better understanding of how adults learn, and personalized feedback to respond to 
individual teacher needs. For example, some supervisors wrote the following: 


This experience really allowed her to grow as we are integrating technology into our curriculum. 
She was able to have discussions with teachers around how students learn, and how technology 
is integrated into the curriculum for student learning, not just experience or exposure. 


The MPA did help [Name] have more difficult conversations with staff regarding student 
learning. 


[Name] has grown in her abilities to adapt to individual teacher needs. 


We are a district moving towards Personalized Learning, so the experience of the MPA on our 
principals’ ability to move their respective staff members forward has been invaluable. 


An additional change noted by supervisors in the area of transformational leadership is that leaders who 
participated in the MPA have been able to foster a culture of collaboration in their schools (n=5). As these 
supervisors explained: 


Our leader has learned how to facilitate change in their particular staff in a collaborative 
manner. 


Current participant has been able to utilize developed skills to turn a new assignment ina 
negative school into a newly energized forward looking school who is again focusing on student 
growth and not on their personal worries. 


Our school was impacted by how our school administrator created a culture of collaboration. 
This improved because it was intentional and our teachers had a better plan to improve their 
communication with one another. 


[Name]...has grown significantly through the MPA. She has been thoughtful in her reflection and 
she has brought her teacher leadership team alongside her in her learning journey....Her staff are 
right on board - this experience is felt and observed when you enter her building. 


Expanded or improved leadership network. Another area of impact also noted by 37% of supervisors 
(n=14) was that school leaders who participated in the MPA have been able to expand or improve their 
leadership network. More specifically, some participants were described as being better connected with other 
principals and school leaders who participated in the MPA, especially those beyond their school (n=10). For 
example: 


Our principal made several great connections to other educational leaders in the state and has 
benefitted from the networking opportunities. 
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The interactions and exchanges conversationally with colleagues enrolled in the MPA and in 
similar roles and positions doing their best to provide exemplary educational opportunities for 
students always has merit and creates lasting relationships. 


The cohort has been an exceptional experience for them to discuss and interact with other school 
leaders that extend beyond our building. 


In a very small school like ours where the principal serves in many capacities, | think the 
networking with other principals was one of the most critical features of the cohort. 


Supervisors also explained that their participants are now better connected with other MPA participants from 
within their own school or district, in turn fostering greater collaboration or a common language (n=5). As one 
supervisor wrote: 


In addition, having been through the MPA myself as a principal...allows us to have a common 
language and focus regarding the components of effective instructional leadership. 


Other impacts. Three supervisors described additional impacts that did not necessarily fall into the four 
broad themes described above. These supervisors explained that their school leaders who attended the MPA 
now have a renewed passion or sense of purpose to their work (n=2), that they are confident the participant will 
be a leader in their district work moving forward (n=1), and that the leader has shifted from being a “manager” 
to a “leader” (n=1). For example, some of these supervisors wrote the following: 


My two principals have made the "mental shift" from manager to leader. | believe the 
combination of formal material in combination of peer interaction has provided a unique 
opportunity for deep reflection and renewed purpose. 


It has been fun to watch her find a new passion after [many] years as a principal. 


Participation in MPA presented challenges or led to minimal changes. While the majority of 
respondents described positive changes as a result of their leaders participating in the MPA, 5 supervisors 
described how the leaders’ participation in the MPA either presented some challenges for the district (n=4) or 
that the leaders’ did not make any significant changes as a result of their participation (n=1). The challenge 
noted by these supervisors was that the time commitment required of leaders placed a burden on the system 
and teachers, at times. As these supervisors explained: 


| think the time commitment is rather extensive and creates perhaps too many days out of the 
office for the administrator to perform regular, routine, and on-going on-site leadership duties. 


The time out of the building is difficult on a small organization. 


In addition, one respondent explained that she did not perceive that the MPA participants she supervised were 
making significant changes in their leadership or their plans as a result of their participation, acknowledging that 
more was expected from them given their time away from their schools. 


Variation across participants. Respondents were also given the opportunity to explain if they believed 
that the impact of participating in the MPA varied across the participants they supervise. The following 
observations were described by a few supervisors: MPA participants who fully engaged in the program appeared 
to gain more from participation; some participants approached the work slightly differently from one another 
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(e.g., an “action-oriented” approach vs. a “more systemic” approach); the program appeared to have the 
greatest impact on the newest/least experienced principals; differences were observed between the district’s 
MPA participants and non-MPA participants; or the impact varied across participants, but this was likely due to 
factors “larger” than the MPA program, such as relationships and personalities. 


Likelihood of future leadership team participation. Respondents were asked to indicate the likelihood 
that they would encourage additional members of their leadership team to participate in the MPA in the future, 
using a 4-point scale from very unlikely to very likely. Of the 45 respondents who answered this item, 62% were 
likely or very likely to encourage others to participate, although 4% were unlikely and 33% were very unlikely to 
do so (see Figure 2). We note that the very unlikely responses may not necessarily reflect dissatisfaction with the 
program but perhaps that many of the leaders in some districts have already participated in the MPA; however, 
we offer this as speculation given that this finding does not align with the otherwise positive ratings to the rest 
of the survey. 


0% 20% 40% 60% 80% 100% 
Very unlikely Unlikely Likely m Very likely 


Figure 2. Likelihood of Encouraging Others to Participate in the MPA 


Additional feedback. Finally, respondents were given the opportunity to provide additional comments 
that they may have about the MPA, and the feedback was largely positive. Supervisors generally shared that the 
program and/or the content of the program was highly valuable and that the program benefitted from good 
leadership and facilitators. In addition, one supervisor shared that it would have further strengthened the 
participants’ work to have the full district team and superintendent attend the MPA, but acknowledged that this 
probably would not have been feasible. Finally, a suggestion offered by one respondent was to incorporate a 
module on teacher evaluations, including how principals can respond to teachers who are not viewed as 
“effective.” Examples of feedback from supervisors about these themes includes the following: 


You might consider creating a stronger focus on teacher evaluation. Specifically, the data show a 
strong trend of principals not being able to terminate ineffective teachers - and we're scared to 
talk about that as professional leaders. The premise that 99% of our teachers are performing at 
a proficient level is ridiculous in any profession much less one as complex as teaching - yet it 
seems no principals know how to tackle that....With your research capabilities | think this could 
be a key learning "module" for principals. 


| believe very strongly in the MPA. Without a doubt, it is the most comprehensive principal 
leadership professional development available for our administrative leaders across the State of 
Minnesota. | have watched our administrative leadership team really grow as a result of their 
learning through MPA. 


The content, the collaborative learning, and having to complete an action learning project have 
made a tremendous impact for our administrators. 


| believe this program should be a requirement of all principals! 
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Summary 


The purpose of this evaluation was to describe and understand the MPA as a resource and training program for 
school leaders. The evaluation was designed to provide information regarding the extent to which participants 
gained knowledge and skills, or changed their behaviors, from the beginning of the program to the end of the 
program in the areas covered by the MPA curriculum. To do this, we administered a retrospective pre/post 
survey to MPA participants. We also surveyed participants’ supervisors to ask them about changes they were 
seeing in participants’ knowledge, skills, or behaviors since participating in the MPA. 


Across both surveys, MPA participants and supervisors shared that participating in the program helped leaders 
to improve their strategic leadership skills, as well as their practices as instructional leaders and 
transformational leaders. In addition, survey results indicate that MPA participants valued the network they 
developed and the resources made available through their participation. An area that emerged as a challenge in 
some cases was that principals’ time out of their buildings to participate in the MPA could be a burden on 
schools and districts, especially in rural or smaller districts. 


In summary, the results from the MPA Participant and Supervisor Surveys were overwhelmingly positive and 
described a pattern of change for the participants in the key areas identified by the MPA curriculum. One 
supervisor respondent stated that, “This program is a necessity for all principals in the state. It provides a 
common understanding of best educational practices, educational research, and a support network among 
administrative colleagues.” 
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Appendix A: MPA Participant Survey® 


Dear MN Principals Academy Participant, 


Now that you are nearing the end of your participation in the MN Principals Academy, we are asking you to 
complete this survey as part of the CAREI evaluation. This online survey should take no more than 15-20 minutes 
to complete. The survey questions are designed to obtain your perspectives and attitudes about a variety of 
topics that you have covered as a participant in the MN Principals Academy. This survey also includes several 
open-ended questions that will allow you to provide valuable insight and feedback. 


By completing the survey, you are indicating that you understand that your participation is voluntary, that you 
may skip any survey question that you choose, and that you agree to participate in this evaluation. Your 
responses will be combined with the responses of others and no names will be used in any reporting. Your 
decision about whether or not to participate will not affect your relationship with the MN Principals Academy or 
the University of Minnesota. 


Thank you for taking the time to complete this survey! 


If you have questions about the survey or the evaluation, please feel free to contact Jane Fields 
(jfields@umn.edu) or Kristin Peterson (kapeters@umn.edu). 


1. What is your current position? 


Superintendent 
Principal 
Assistant Principal 
District-Level Department Director 
Education Director 
Dean of Students 
Other (please specify) 


OCO0O0000 


In the sections that follow, you will be asked to compare your district, your work, or your teachers prior to and 
after your participation in the MN Principals Academy. For each item: 


1. Rate your level of agreement to that item prior to your participation in the MN Principals Academy in the 
"Before the program" column. 


2. Rate your level of agreement to that item after your participation in the MN Principals Academy in the "After 
the program" column. 


8 The survey items presented here do not always follow in the same order as presented in the report above. 
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2. To what extent do you agree or disagree with these statements about your district? 


Before the program 


After the Program 


strengly Disagree | Agree Sieriely 
disagree agree 


Strongly 


disagree DIsaetee 


Agree 


Strongly 
agree 


a. District staff help to ensure that our schools offer an 
aligned curriculum. 


b. Instructional leadership is a key component of our 
principal evaluation system. 


c. District leaders often discuss instructional issues with 
school leadership teams. 


d. Our district offers a range of differentiated safety 
nets or interventions available at the individual student 
level. 


e. Our district has strategies to support principals and 
teachers in preparing students for college or other post- 
secondary education beyond high school. 


f. District administrators model high levels of 
professional practice. 


3. To what extent do you agree or disagree with these statements about your distr 


Before the program 


strengly Disagree | Agree SvOnely 
disagree agree 


ict? 


Strongly 


disagree Diseetce 


Agree 


After the Program 


Strongly 
agree 


a. Our district culture is one in which all teachers and 
administrators feel responsible for working together to 
improve student achievement. 


b. Our district has institutionalized processes that 
ensure the professional development program is 
aligned with a belief that all students can achieve the 
same high standards. 


c. Principals participate in ongoing high-quality 
professional development. 


d. Teaching is aligned to rigorous performance 
standards. 


e. Our district supports equity-related initiatives. 
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4. To what extent do you agree or disagree with the following statements as they relate to your work? 


Before the program 


After the Program 


strongly Disagree | Agree Strongly 
disagree agree 


Strongly 


disagree Disaeree 


Agree 


Strongly 
agree 


a. | have established a clear vision for my school. 


b. | have established an effective data management 
system to collect student performance data 
continuously throughout the year. 


c. lam comfortable defining for my teachers what 
constitutes effective standards for instructional 
practice. 


d. | expect teachers to design lessons by working 
backward from the standards and assessments. 


e. | ensure that our school's safety nets and 
interventions are successful in getting struggling 
students back on track. 


f. | understand the seminal research on "how people 
learn." 


Before the program 


strongly Disagree | Agree sone 
disagree agree 


After the Program 


Strongly 


disagree Disaefce 


5. To what extent do you agree or disagree with the following statements as they relate to your work? 


Agree 


Strongly 
agree 


a. | discuss why student depth of understanding is 
critical to achieving our vision. 


b. | ensure that the school has a standards-based 
instructional system that is aligned with high 
performance standards. 


c. | am clear in my expectation that teachers will 
regularly meet to assess student work against the 
standards. 


d. | convey to teachers how important it is for 
classroom assessments to be carefully aligned to 
content standards. 


e. | ensure that my staff understands the seminal 
research on how people learn to better instruct 
students in their classrooms. 
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6. To what extent do you agree or disagree with the following statements as they relate to your work? 


Before the program 


After the Program 


Strongly 


disagree Disaeree 


Agree 


Strongly 
agree 


Strongly 


disagree Disaeree 


Agree 


Strongly 
agree 


a. | monitor the effectiveness of instructional practice in 
the school. 


b. | insist that the way to improve student learning is 
through improvement of instructional practices. 


c. | challenge teachers to try innovative evidence-based 
instructional approaches. 


d. |am able to motivate teachers who are reluctant to 
consider new instructional practices. 


e. | search widely to benchmark our school’s 
performance and identify strategies that have enabled 
high-performing schools to sustain improvement. 


f. | encourage and support innovative improvement 
practices such as coaching, modeling, observing 
practice, and providing feedback. 


g. | make judgments about teacher performance using a 
variety of data sources, including student results. 


h. In my school, | have institutionalized processes that 
ensure that the professional development system is 
aligned with a belief that all students can achieve the 
same high standards. 


i. | have strategies to use in order to buffer teachers 
from distractions to their instruction. 
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7. To what extent do you agree or disagree with the following statements about your school? 


Before the program 


After the Program 


Strongly 
disagree 


Disagree 


Agree 


Strongly 
agree 


Strongly 


disagree Disagree | Agree 


Strongly 
agree 


a. We provide an aligned curriculum for students across 
the grades. 


b. There is a strong commitment in this school to a 
common set of shared goals. 


c. Our school improvement plan drives teachers’ 
professional development. 


d. Administrators and teachers collectively plan who 
will provide leadership for initiatives. 


e. The schedule provides adequate time for 
collaborative teacher planning. 


f. We have a range of differentiated safety nets 
available at the individual student level. 


g. Most of the parents of students in this school expect 
their children to go on to some sort of post-secondary 
education. 


h. Team leadership at all levels focuses on the guidance, 
direction, and support of sustained improvement in 
instructional practice and student learning. 


8. To what extent do you agree or disagree that teachers in your school... 


Before the program 


Strongly 
disagree 


Disagree 


Agree 


Strongly 
agree 


After the Program 


Strongly 


isderee Disagree | Agree 


Strongly 
agree 


a. Are able to accurately monitor the progress of their 
students? 


b. Have a sense of collective responsibility for student 
learning? 


c. Have the knowledge and skills they need to improve 
student learning? 


d. Have ongoing conversations among themselves 
about instructional practices? 


e. Prepare all students to go on to some sort of post- 
secondary education? 


f. View problems as issues to be solved, not as barriers 
to action? 
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9. How often do you... 


Before the program After the Program 


Never | Seldom | Sometimes |Regularly; Never Seldom |Sometimes |Regularly 


a. Discuss instructional issues with your 
teachers? 


b. Discuss equity issues with your teachers? 


c. Visit classrooms to briefly observe instruction? 


d. Watch an entire lesson when observing 
classroom instruction? 


e. Model instructional strategies for teachers? 


f. Give teachers specific ideas for how to improve 
their instruction? 


g. Ask teachers about their use of data in 
instructional decision-making? 


10. How often do you... 


Before the program After the Program 


Never | Seldom |Sometimes|Regularly; Never Seldom |Sometimes |Regularly 


a. Make judgments about teacher performance 
based on student-level data? 


b. Use research evidence in your decision- 
making? 


c. Attend teacher professional learning 
community meetings? 


d. Provide or locate resources to help staff 
improve their teaching? 
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11. How often do... 


Before the program 


After the Program 


More than} Monthly| Several | Annually 
once a times a 
month year 


More than 
once a 
month 


Monthly} Several | Annually 
times a 
year 


a. You communicate to the community your 
commitment to the achievement of high performance 
standards by all students? 


b. You rely on systematically collected evaluation data 
about your school in decision making? 


c. Teachers observe each other's classrooms? 


12. Please rate your own level of expertise in the following areas. 


Before the program 


After the Program 


Highly 


Basic | Moderate] Good 
Developed 


Basic |Moderate} Good 


Highly 
Developed 


a. Developing teamwork among teachers 


b. Developing teacher leaders 


c. Motivating teachers who are reluctant to consider 
new instructional practices 


d. Generating enthusiasm for a shared vision for the 
school 


e. Analyzing and interpreting student assessment data 


f. Using student achievement data to help make 
decisions 


g. Solving problems systematically by examining the 
whole picture, rather than isolated parts 


13. What is your greatest “take-away” from your participation in the Minnesota Principals Academy? (Please 


name/discuss only one.) 


14. What content (unit or otherwise) did you find to be the most helpful to you, and why? 


15. What are you doing differently now, if anything, in your school as a result of this experience? How do you 


see it impacting teachers and/or students? 
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Appendix B: MPA Supervisor Survey? 


Dear District Leader, 


Now that one or more of the people who you supervise are nearing the end of their participation in the 
Minnesota Principals Academy, we are asking you to complete this survey as part of the University of 
Minnesota’s Center for Applied Research and Educational Improvement’s evaluation of the program. This online 
survey should take no more than 10-15 minutes to complete. The survey questions are designed to gather 
feedback about the ways in which the Minnesota Principals Academy has impacted the participant(s) and your 
district. Your responses will remain confidential, and results will be reported in such a way that no individual 
respondent will be identifiable. 


By completing the survey, you are indicating that you understand that your participation is voluntary, that you 
may skip any survey question that you choose, and that you agree to participate in this evaluation. Your decision 
about whether or not to participate will not affect your relationship with the Minnesota Principals Academy or 
the University of Minnesota. 


If you have questions about the survey or the evaluation, please feel free to contact Jane Fields 
(jfields@umn.edu) or Kristin Peterson (kapeters@umn.edu). 


Thank you for your time! 


When completing this survey, please think about the person in your school/district who you currently supervise 
and who is currently participating in the Minnesota Principals Academy. If multiple people who you supervise in 
your school or district are participating, please think in general about all of them when answering these survey 
items. 


1. As a result of participating in the MPA, the participant(s) has improved their... 


Strongly 
Agree 


Strongly 


Disagree Disagree Agree 


a. Leadership skills. 


b. Team-building skills (e.g., he/she promotes professional learning with 
staff members, shares leadership, supports ideas offered by team 
members). 


c. Ability to motivate teachers or other staff members who are reluctant 
to consider new instructional practices. 


d. Commitment to effort-based theories of learning where all students 
can learn. 


e. Ability to think more systemically about how to improve instruction. 


f. Ability to recognize the individual needs of their staff members and to 
support them accordingly. 


° The survey items presented here do not always follow in the same order as presented in the report above. 
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2. As a result of participating in the MPA, the participant(s) has improved their ability to... 


a. Ensure that adequate resources are prioritized and allocated to the initiatives 
that will achieve maximum benefit in improving learning for all students. 


Strongly 
Disagree 


Disagree 


Agree 


Strongly 
Agree 


b. Ensure that teachers use data to make instructional decisions for students. 


c. Create a compelling school vision in which all students are held to high 
expectations. 


d. Regularly monitor and analyze a range of student performance data to make 
judgments about areas of curriculum and instruction that need attention. 


e. Identify research and best practices that support instructional improvement. 


f. Listen to community members, advocates, or other stakeholders to inform 
change within the school. 


3. In what ways, if any, has participation in the MPA by one or more of your leaders impacted your district or 
specific schools in your district? (Please provide specific examples, if possible). 


4. If the impact of participating in the MPA varied across the participants you supervise, please feel free to 


elaborate how it was different in the space below. 


5. In the future, what is the likelihood that you would encourage additional members of your leadership team 


to participate in the MPA? 
O Very unlikely 
O Unlikely 
O Likely 
O Very likely 


6. Please provide any other comments you may have about the MPA. 
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Appendix C: MPA Participant Survey Results — Brainerd/Staples Cohort 


Table C1. Brainerd/Staples: Current Position 


n % 
School-level Positions 30 83% 
Principal / Charter School Executive Director 24 67% 
Assistant Principal 5 14% 
Other School Leaders 1 3% 
District-level Positions 4 11% 
Superintendent - - 
Assistant Superintendent A 3% 
Department Director 3 8% 
Other 2 6% 
Total 36 100% 


Table C2. Brainerd/Staples: Self-Reported Level of Expertise 


Pre/ Highly 

eo) Basic Moderate (efofoye mmm (=\"=1() 1-10) 
a. Developing teamwork among teachers Be a a7 pe ee tes 
‘ Post 29 - 28% 62% 10% 
b. Developing teacher leaders ve cae — ma ate aon 
ene Post 29 3% 17% 59% 21% 
c. Motivating teachers who are reluctant to consider new Pre 29 28% 41% 24% 7% 
instructional practices Post 29 7% 28% 55% 10% 
; P - Pre 29 7% 38% 48% 7% 
d. Generating enthusiasm for a shared vision for the school Post 29 : 14% 72% 14% 
s : , Pre 29 3% 52% 41% 3% 
e. Analyzing and interpreting student assessment data Post 29 39% 31% 59% 7%, 
F : 7 Pre 29 3% 55% 38% 3% 
f. Using student achievement data to help make decisions Post 29 7%, 14% 66% 14% 
g. Solving problems systematically by examining the whole Pre 29 3% 41% 48% 7% 
picture, rather than isolated parts Post 29 3% 14% 69% 14% 


Note. Results include school-level respondents only. 


Table C3. Brainerd/Staples: Self-Reported Instructional Leadership Skills 


Strongly Strongly 

n disagree Disagree Agree Ey4g-{-] 
a. | monitor the effectiveness of instructional practice inthe Pre 30 - 23% 73% 3% 
school. Post 30 3% 23% 63% 10% 
b. linsist that the way to improve student learning is Pre 30 3% 7% 87% 3% 
through improvement of instructional practices. Post 30 - 7% 57% 37% 
c. | challenge teachers to try innovative evidence-based Pre 30 - 23% 73% 3% 
instructional approaches. Post 30 - 13% 50% 37% 
d.! am able to motivate teachers who are reluctant to Pre 30 - 40% 53% 7% 
consider new instructional practices. Post 30 - 17% 70% 13% 
e. | encourage and support innovative improvement Pre 30 5 7% 80% 13% 


practices such as coaching, modeling, observing practice, 


and providing feedback. Post 30 = 3% 50% 47% 
0, 0, 0, : 

f. | understand the seminal research on “how people learn.” an ph eas eae ner, 

g. | ensure that my staff understands the seminal research Pre 30 30% 53% 17% 0% 

on how people learn to better instruct students in their 

classrooms. Post 30 : 30% 63% 7% 


Note. Results include school-level respondents only. 
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Table C4. Brainerd/Staples: Perceptions of Defining and Using Standards 


a. 1am comfortable defining for my teachers what constitutes 
effective standards for instructional practice. 

b. | expect teachers to design lessons by working backward 
from the standards and assessments. 

c. | ensure that the school has a standards-based instructional 
system that is aligned with high performance standards. 

d.! am clear in my expectation that teachers will regularly 
meet to assess student work against the standards. 

e. | convey to teachers how important it is for classroom 
assessments to be carefully aligned to content standards. 


Pre/ 
Post 
Pre 
Post 
Pre 
Post 
Pre 
Post 
Pre 
Post 
Pre 
Post 


n_ disagree Disagree 


Strongly 


43% 
13% 
37% 
13% 
40% 
17% 
50% 
30% 
27% 
17% 


Agree 
53% 
57% 
60% 
63% 
50% 
60% 
47% 
53% 
63% 
60% 


Strongly 
Ey 40-12] 
3% 
30% 
23% 
3% 
20% 
13% 


23% 


Note. Results include school-level respondents only. 


Table C5. Brainerd/Staples: Self-Reported Practices Related to School Vision, Structures, and Management 


Pre/ Strongly Strongly 
Post n_ disagree Disagree Agree agree 

a. | have established a clear vision for my school ne au tue — peas i 

: Post 30 13% 37% 40% 10% 
b. | have established an effective data management system to Pre 30 7% 40% 47% 7% 
collect student performance data continuously throughout 
the year. Post 30 17% 20% 57% 7% 
c. | ensure that our school's safety nets and interventions are Pre 30 3% 37% 53% 7% 
successful in getting struggling students back on track. Post 30 3% 30% 53% 13% 
d. | discuss why student depth of understanding is critical to Pre 30 10% 50% 40% - 
achieving our vision. Post 30 - 27% 53% 20% 
e. | search widely to benchmark our school’s performance and ‘Pre 30 3% 73% 23% ia 
identify strategies that have enabled high-performing schools 
to sustain improvement. Post 30 3% 43% 50% 3% 
f. | make judgments about teacher performance using a Pre 30 = 23% 77% - 
variety of data sources, including student results. Post 30 3% 13% 67% 17% 
g. In my school, | have institutionalized processes that ensure Pre 30 = 63% 33% 3% 
that the professional development system is aligned with a 
belief that all students can achieve the same high standards. Post 30 3% 37% 43% 17% 
h. | have strategies to use in order to buffer teachers from Pre 30 3% 50% 43% 3% 
distractions to their instruction. Post 30 - 37% 50% 13% 

Note. Results include school-level respondents only. 
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Table C6. Brainerd/Staples: Self-Reported Frequency of Interaction with Teachers and Use of Research 
Pre/ 


How often do you... Post n Never Seldom Sometimes Regularly 
a. Discuss instructional issues with your teachers? ae a i Lis ee te 
uals i Z Post 29 - 7% 45% 48% 
Pre 29 10% 41% 45% 3% 

' aes : 5 
b. Discuss equity issues with your teachers? Post 29 ; 238% 18% 24% 
Visit classrooms to briefly observe instruction? ine os a ae Boe ae 
c. Visit classrooms to briefly observe instruction: Post 29 . 17% 34% 18% 
d. Watch an entire lesson when observing classroom Pre 29 - 14% 55% 31% 
instruction? Post 29 - 10% 45% 45% 
Pre 29 17% 48% 34% - 
; : ; = 
e. Model instructional strategies for teachers? Post 29 10% 34% 52% 3% 
f. Give teachers specific ideas for how to improve their Pre 29 - 10% 69% 21% 
instruction? Post 29 - 3% 69% 28% 
g. Ask teachers about their use of data in instructional Pre 29 3% 17% 62% 17% 
decision-making? Post 29 - 3% 45% 52% 
Pre 29 10% 31% 48% 10% 
. : F nee 

h. Attend teacher professional learning community meetings? Post 29 3% 2% 38% 34% 
i. Provide or locate resources to help staff improve their Pre 29 0% 24% 69% 7% 
teaching? Post 29 0% 3% 55% 41% 
j. Made judgements about teacher performance based on Pre 29 3% 24% 62% 10% 
student-level data? Post 29 0% 14% 62% 24% 
k. Use research evidence in your own decision making? Efe 2 ee ie bee Les 
y 2 Post 29 0% 3% 41% 55% 


Note. Results include school-level respondents only. 


Table C7. Brainerd/Staples: Self-Reported Frequency of Community Engagement, Data Use, and Peer Observations 
Pre/ RY=\0-) 1 More than 


How often do... Post n Annually times/yr Monthly once amonth 
a. You communicate to the community your Pre 28 50% 39% 11% - 
commitment to the achievement of high ep . ‘ 
performance standards by all students? Post 28 36% 43% 21% : 
b. You rely on systematically collected evaluation Pre 28 36% 61% - 4% 
data about your school in decision making? Post 28 25% 57% 14% 4% 
c. Teachers observe each other's classrooms? Eke 28 oe aon ee i: 
é : Post 28 39% 50% 4% 7% 


Note. Results include school-level respondents only. 


Table C8. Brainerd/Staples: Perceptions of Teacher Practices 


Strongly Strongly 
disagree Disagree Agree agree 
a. Teachers in this school are able to accurately Pre 30 - 27% 63% 10% 
monitor the progress of their students. Post 30 10% 20% 60% 10% 
b. Teachers in this school have a sense of collective Pre 30 - 33% 63% 3% 
responsibility for student learning. Post 30 - 37% 53% 10% 
c. Teachers in this school have the knowledge and Pre 30 - 30% 63% 7% 
skills they need to improve student learning. Post 30 - 37% 63% - 
d. Teachers in this school have ongoing conversations Pre 30 3% 40% 57% - 
among themselves about instructional practices. Post 30 7% 17% 67% 10% 
e. Teachers in this school prepare all students to go Pre 29 7% 38% 55% - 
on to some sort of post-secondary education. Post 29 - 48% 52% - 
f. Teachers in this school view problems as issues to Pre 29 14% 45% 38% 3% 
be solved, not as barriers to action. Post 30 7% 40% 47% 7% 


Note. Results include school-level respondents only. 
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Table C9. Brainerd/Staples: Perceptions of School-Level Practices 


Strongly 


Strongly 


agree 


a. We provide an aligned curriculum for students across the 
grades. 

b. There is a strong commitment in this school to a 
common set of shared goals. 

c. Our school improvement plan drives teachers’ 
professional development. 

d. Administrators and teachers collectively plan who will 
provide leadership for initiatives. 

e. The schedule allows for adequate embedded time for 
collaborative teacher planning. 

f. We have a range of differentiated safety nets available at 
the individual student level. 

g. Team leadership at all levels focuses on the guidance, 
direction, and support of sustained improvement in 
instructional practice and student learning. 

h. Most parents of students in this school expect their 
children to go on to some sort of post-secondary 
education. 


disagree 


Disagree 


47% 
17% 
47% 
23% 
40% 
27% 
40% 
20% 
33% 
30% 
47% 
30% 
57% 


37% 
37% 
33% 


Agree 


50% 
70% 
47% 
60% 
53% 
47% 
53% 
63% 
40% 
47% 
50% 
53% 
40% 


57% 
53% 
63% 


Note. Results include school-level respondents only. 


Table C10. Brainerd/Staples: Perceptions of District-Level Practices 
Pre/ 


Post 


n 


Strongly 
disagree 


Disagree 


Agree 


Strongly 
agree 


a. District staff help to ensure that our schools offer an 
aligned curriculum. 

b. Instructional leadership is a key component of our 
principal evaluation system. 

c. District leaders often discuss instructional issues with 
school leadership teams. 

d. Our district offers a range of differentiated safety nets or 
interventions available at the individual student level. 

e. Our district has strategies to support principals and 
teachers in preparing students for college or other post- 
secondary education beyond high school. 

f. District administrators model high levels of professional 
practice. 

g. Our district culture is one in which all teachers and 
administrators feel responsible for working together to 
improve student achievement. 

h. Our district has institutionalized processes that ensure 
the PD program is aligned with a belief that all students 
can achieve the same high standards. 

i. Principals participate in ongoing high-quality professional 
development. 


j. Teaching is aligned to rigorous performance standards. 


k. Our district supports equity-related initiatives. 


Pre 
Post 
Pre 
Post 
Pre 
Post 
Pre 
Post 
Pre 


Post 


Pre 
Post 
Pre 


Post 
Pre 
Post 


Pre 
Post 
Pre 
Post 
Pre 
Post 


34 
34 
34 
34 
33 
33 
34 
34 
34 


34 


34 
34 
34 


34 
34 
32 


34 
34 
34 
34 
34 
34 


71% 
79% 
59% 
35% 
48% 
39% 
26% 
35% 
76% 


71% 
79% 
62% 
41% 
41% 
44% 
38% 


35% 
56% 
50% 
53% 
53% 
44% 


Note. Results include both school-level and district-level respondents. 
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Appendix D: MPA Participant Survey Results — Fergus Falls Cohort 


Table D1. Fergus Falls: Current Position 


n % 
School-level Positions 26 90% 
Principal / Charter School Executive Director 21 72% 
Assistant Principal 3 10% 
Other School Leaders 2 7% 
District-level Positions = = 
Superintendent - - 
Assistant Superintendent - - 
Department Director - - 
Other 3 10% 
Total 29 100% 


Table D2. Fergus Falls: Self-Reported Level of Expertise 


Pre/ Highly 
Post n Basic Moderate (cfofole Mmmm (=\17-1(o)el-Le| 
a. Developing teamwork among teachers eG ae ss ue Sas ue 
Post 25 - 8% 84% 8% 
b. Developing teacher leaders i ada Pee ve : 
Post 25 8% 12% 72% 8% 
c. Motivating teachers who are reluctant to consider new Pre 25 40% 44% 16% - 
instructional practices Post 25 - 48% 44% 8% 
F F ae Pre 25 20% 52% 24% 4% 
d. Generating enthusiasm for a shared vision for the school Post 25 228% 68% A% 
e. Analyzing and interpreting student assessment data nice ee ie au se ; 
Post 25 - 20% 80% - 
F : a Pre 25 12% 48% 40% - 
f. Using student achievement data to help make decisions Post 25 ; 12% 84% AY% 
g. Solving problems systematically by examining the whole Pre 25 40% 44% 16% - 
picture, rather than isolated parts Post 25 - 36% 60% 4% 


Note. Results include school-level respondents only. 


Table D3. Fergus Falls: Self-Reported Instructional Leadership Skills 


Strongly Strongly 

disagree Disagree agree 
a. | monitor the effectiveness of instructional practice inthe Pre 25 4% 20% 72% 4% 
school. Post 25 - 8% 60% 32% 
b. l insist that the way to improve student learning is Pre 25 - 24% 68% 8% 
through improvement of instructional practices. Post 25 - 4% 48% 48% 
c. | challenge teachers to try innovative evidence-based Pre 25 - 52% 40% 8% 
instructional approaches. Post 25 - 8% 48% 44% 
d.! am able to motivate teachers who are reluctant to Pre 25 8% 48% 44% - 
consider new instructional practices. Post 25 - 16% 64% 20% 
e. | encourage and support innovative improvement Pre 25 = 16% 76% 8% 
practices such as coaching, modeling, observing practice, 
and providing feedback. Post 25 ° A 44% 56% 

' ti » Pre 25 20% 64% 12% 4% 

f. | understand the seminal research on “how people learn. Post 25 - - 68% 32% 
g. | ensure that my staff understands the seminal research Pre 25 16% 76% 8% = 
on how people learn to better instruct students in their 
claccroome. Post 25 - 28% 60% 12% 


Note. Results include school-level respondents only. 
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Table D4. Fergus Falls: Perceptions of Defining and Using Standards 


Pre/ Strongly Strongly 

Post n_ disagree Disagree Agree agree 
a. 1am comfortable defining for my teachers what constitutes Pre 25 = 60% 32% 8% 
effective standards for instructional practice. Post 25 - 4% 52% 44% 
b. | expect teachers to design lessons by working backward Pre 25 - 36% 56% 8% 
from the standards and assessments. Post 25 - 4% 68% 28% 
c. | ensure that the school has a standards-based instructional Pre 25 4% 20% 72% 4% 
system that is aligned with high performance standards. Post 25 12% 68% 20% 
d.!am clear in my expectation that teachers will regularly meet Pre 25 - 64% 28% 8% 
to assess student work against the standards. Post 25 . 24% 44% 32% 
e. | convey to teachers how important it is for classroom Pre 25 - 56% 32% 12% 
assessments to be carefully aligned to content standards. Post 25 - 12% 48% 40% 


Note. Results include school-level respondents only. 


Table D5. Fergus Falls: Self-Reported Practices Related to School Vision, Structures, and Management 


Pre/ Strongly Strongly 
Post n_ disagree Disagree Agree Ey4g-\-] 
9 ie) 9 
a. | have established a clear vision for my school. a a : et jen oe 
b. | have established an effective data management system to Pre 25 A% 60% 36% iz 
collect student performance data continuously throughout 
thavear Post 25 8% 20% 64% 8% 
y 

c. | ensure that our school's safety nets and interventions are Pre 25 - 48% 48% 4% 
successful in getting struggling students back on track. Post 24 - 21% 63% 17% 
d. | discuss why student depth of understanding is critical to Pre 25 12% 60% 24% 4% 
achieving our vision. Post 25 - 8% 72% 20% 
e. | search widely to benchmark our school’s performance and Pre 25 16% 60% 20% 4% 
identify strategies that have enabled high-performing schools 

wa Post 25 - 56% 28% 16% 
to sustain improvement. 
f. | make judgments about teacher performance using a Pre 25 - 44% 48% 8% 
variety of data sources, including student results. Post 25 - 16% 52% 32% 
g. In my school, | have institutionalized processes that ensure Pre 25 Q% 48% 36% 3% 
that the professional development system is aligned with a 
belief that all students can achieve the same high standards. Post 25 4% 8% 64% 24% 
h. | have strategies to use in order to buffer teachers from Pre 25 16% 56% 24% 4% 
distractions to their instruction. Post 25 4% 20% 68% 8% 


Note. Results include school-level respondents only. 
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Table D6. Fergus Falls: Self-Reported Frequency of Interaction with Teachers and Use of Research 
Pre/ 


How often do you... Post n Never’ Seldom Sometimes Regularly 
a. Discuss instructional issues with your teachers? Me oe ae ae ess 20% 
Batu 2 Z Post 25 - : 36% 64% 
Pre 25 4% 68% 24% 4% 

' aes : 5 
b. Discuss equity issues with your teachers? Post 5 A% 12% 60% 24% 
Visit classrooms to briefly observe instruction? ile oe : ane Re rates 
c. Visit classrooms to briefly observe instruc ? Post 25 7 i 8% 52% 
d. Watch an entire lesson when observing classroom Pre 25 8% 12% 48% 32% 
instruction? Post 25 - 8% 44% 48% 
Pre 25 8% 52% 36% 4% 
; : ; = 

e. Model instructional strategies for teachers? Post 25 A%, AA% 32% 20% 
f. Give teachers specific ideas for how to improve their Pre 25 4% 12% 64% 20% 
instruction? Post 25 - 4% 44% 52% 
g. Ask teachers about their use of data in instructional Pre 25 8% 32% 48% 12% 
decision-making? Post 25 - 16% 44% 40% 
9 9 9 9 
h. Attend teacher professional learning community meetings? Sie - 5 ce si re 
a ‘0 ‘0 ‘0 
i. Provide or locate resources to help staff improve their Pre 25 4% 24% 32% 40% 
teaching? Post 25 - 8% 28% 64% 
j. Made judgements about teacher performance based on Pre 25 8% 28% 48% 16% 
student-level data? Post 25 - 12% 60% 28% 
k. Use research evidence in your own decision making? Ee gop ee eee 28% uae 
cae y 2 Post 25 - 16% 48% 36% 


Note. Results include school-level respondents only. 


Table D7. Fergus Falls: Self-Reported Frequency of Community Engagement, Data Use, and Peer Observations 
Several More than 


Annually _ times/yr Monthly once a month 

a. You communicate to the community your Pre 25 56% 40% 4% = 
commitment to the achievement of high Post 25 28% 60% 8% A% 
performance standards by all students? 
b. You rely on systematically collected evaluation Pre 25 40% 52% 8% - 
data about your school in decision making? Post 25 16% 56% 28% - 

Pre 25 72% 28% - - 

' 2 

c. Teachers observe each other's classrooms? Post 25 10% 60% : 


Note. Results include school-level respondents only. 


Table D8. Fergus Falls: Perceptions of Teacher Practices 


Strongly Strongly 
disagree Disagree Agree Ey4g-{-) 
a. Teachers in this school are able to accurately Pre 25 4% 24% 60% 12% 
monitor the progress of their students. Post 25 - 16% 60% 24% 
b. Teachers in this school have a sense of collective Pre 25 - 32% 64% 4% 
responsibility for student learning. Post 25 - 4% 76% 20% 
c. Teachers in this school have the knowledge and Pre 25 - 44% 52% 4% 
skills they need to improve student learning. Post 25 - 16% 68% 16% 
d. Teachers in this school have ongoing conversations Pre 25 12% 40% 40% 8% 
among themselves about instructional practices. Post 25 - 16% 68% 16% 
e. Teachers in this school prepare all students to go Pre 25 4% 28% 68% - 
on to some sort of post-secondary education. Post 25 - 20% 76% 4% 
f. Teachers in this school view problems as issues to Pre 25 12% 56% 32% - 
be solved, not as barriers to action. Post 25 - 28% 60% 12% 


Note. Results include school-level respondents only. 
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Table D9. Fergus Falls: Perceptions of School-Level Practices 


Pre/ Strongly 
Post n disagree Disagree 
a. We provide an aligned curriculum for students Pre 25 = 32% 
across the grades. Post 25 4% 28% 
b. There is a strong commitment in this school to a Pre 25 - 60% 
common set of shared goals. Post 25 - 28% 
c. Our school improvement plan drives teachers’ Pre 25 4% 60% 
professional development. Post 25 4% 24% 
d. Administrators and teachers collectively plan who Pre 25 8% 44% 
will provide leadership for initiatives. Post 25 - 20% 
e. The schedule allows for adequate embedded time Pre 25 12% 28% 
for collaborative teacher planning. Post 25 8% 16% 
f. We have a range of differentiated safety nets Pre 25 8% 48% 
available at the individual student level. Post 25 12% 24% 
g. Team leadership at all levels focuses on the Pre 5 12% 64% 
guidance, direction, and support of sustained 
aprOvelnent in instructional practice and student Post 25 - 36% 
learning. 
h. Most parents of students in this school expect their Pre 25 - 20% 
children to go on to some sort of post-secondary Post 25 : 20% 


education. 


Agree 


56% 
40% 
28% 
52% 
28% 
52% 
40% 
60% 
48% 
56% 
40% 
60% 


20% 


52% 


80% 
76% 


Strongly 


agree 
12% 
28% 
12% 
20% 
8% 
20% 
8% 
20% 
12% 
20% 
4% 
4% 
4% 


12% 


4% 


Note. Results include school-level respondents only. 


Table D10. Fergus Falls: Perceptions of District-Level Practices 


Disagree 


Agree 


Strongly 
agree 


Strongly 
disagree 
a. District staff help to ensure that our schools offer an Pre 25 - 
aligned curriculum. Post 25 - 
b. Instructional leadership is a key component of our Pre 25 - 
principal evaluation system. Post 25 8% 
c. District leaders often discuss instructional issues with Pre 25 4% 
school leadership teams. Post 25 8% 
d. Our district offers a range of differentiated safety nets or Pre 25 4% 
interventions available at the individual student level. Post 25 8% 
e. Our district has strategies to support principals and Pre 25 a 
teachers in preparing students for college or other post- 
secondary education beyond high school. POR Ce oe 
f. District administrators model high levels of professional Pre 25 - 
practice. Post 25 4% 
g. Our district culture is one in which all teachers and Pre 25 4% 
administrators feel responsible for working together to 
Post 25 - 


improve student achievement. 

h. Our district has institutionalized processes that ensure Pre 25 = 
the PD program is aligned with a belief that all students 
can achieve the same high standards. 

i. Principals participate in ongoing high-quality professional Pre 25 - 


Post 23 4% 


development. Post 25 4% 

j. Teaching is aligned to rigorous performance standards fi oe : 
J plea cpa "Post 24 «= 4% 
ears ; Penne Pre 25 - 

k. Our district supports equity-related initiatives. Post 25 - 


20% 
16% 
12% 
12% 
64% 
32% 
24% 
20% 
28% 
24% 
20% 
16% 
36% 
32% 
48% 
35% 
60% 
24% 
44% 
21% 
52% 
32% 


64% 
68% 
76% 
40% 
28% 
52% 
64% 
64% 
64% 
60% 
72% 
44% 
52% 
52% 
52% 
52% 
28% 
20% 
48% 
58% 
40% 
64% 


Note. Results include both school-level and district-level respondents. 
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Appendix E: MPA Participant Survey Results — Twin Cities Cohort 


Table E1. Twin Cities: Current Position 


n % 
School-level Positions 25 93% 
Principal / Charter School Executive Director 15 56% 
Assistant Principal 9 33% 
Other School Leaders 1 4% 
District-level Positions 1 4% 
Superintendent 1 4% 
Assistant Superintendent - - 
Department Director - - 
Other 1 4% 
Total 27 100% 
Table E2. Twin Cities: Self-Reported Level of Expertise 
Pre/ Highly 
A) Basic Moderate Good developed 
a. Developing teamwork among teachers es = gee sae ae ee 
b. Developing teacher leaders ig oe rae sabes ae me 
Post 25 - 20% 44% 36% 
c. Motivating teachers who are reluctant to consider new Pre 25 28% 48% 24% - 
instructional practices Post 25 4% 24% 68% 4% 
. ; ea Pre 25 20% 52% 24% 4% 
d. Generating enthusiasm for a shared vision for the school Post 25 - A% 76% 20% 
e. Analyzing and interpreting student assessment data oe - ve ye ae ie 
. ; -_ Pre 25 12% 20% 60% 8% 
f. Using student achievement data to help make decisions Post 25 : 12% 56% 32% 
g. Solving problems systematically by examining the whole Pre 25 12% 40% 40% 8% 
picture, rather than isolated parts Post 25 - 8% 44% 48% 


Note. Results include school-level respondents only. 


Table E3. Twin Cities: Self-Reported Instructional Leadership Skills 


Strongly Strongly 
n disagree Disagree Agree Eyig-{-) 

a. | monitor the effectiveness of instructional practice inthe Pre 25 4% 16% 76% 4% 
school. Post 25 - 4% 52% 44% 
b. linsist that the way to improve student learning is Pre 25 4% 16% 64% 16% 
through improvement of instructional practices. Post 25 - 4% 16% 80% 
c. | challenge teachers to try innovative evidence-based Pre 25 - 28% 68% 4% 
instructional approaches. Post 25 - - 56% 44% 
d.! am able to motivate teachers who are reluctant to Pre 25 8% 48% 44% - 
consider new instructional practices. Post 25 - - 80% 20% 
e. | encourage and support innovative improvement Pre 24 = 17% 75% 8% 
practices such as coaching, modeling, observing practice, 
and providing feedback. Robs ae Z ” 28 oe 
f. | understand the seminal research on “how people learn.” ne a ane eila pa : 

Post 25 - - 40% 60% 
g. | ensure that my staff understands the seminal research Pre 25 24% 64% 12% - 
on how people learn to better instruct students in their 
Fareroame: Post 25 4% 28% 48% 20% 


Note. Results include school-level respondents only. 
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Table E4. Twin Cities: Perceptions of Defining and Using Standards 


Pre/ Strongly Strongly 

Post n_ disagree Disagree Agree agree 
a. 1am comfortable defining for my teachers what constitutes Pre 25 4% 36% 56% 4% 
effective standards for instructional practice. Post 25 - 4% 48% 48% 
b. | expect teachers to design lessons by working backward from Pre 25 4% 24% 60% 12% 
the standards and assessments. Post 25 8% 4% 44% 44% 
c. | ensure that the school has a standards-based instructional Pre 25 4% 32% 56% 8% 
system that is aligned with high performance standards. Post 25 - 8% 52% 40% 
d.!am clear in my expectation that teachers will regularly meet Pre 25 4% 52% 40% 4% 
to assess student work against the standards. Post 25 4% 16% 28% 52% 
e. | convey to teachers how important it is for classroom Pre 25 4% 40% 52% 4% 
assessments to be carefully aligned to content standards. Post 25 - 8% 48% 44% 


Note. Results include school-level respondents only. 


Table E5. Twin Cities: Self-Reported Practices Related to School Vision, Structures, and Management 


Pre/ Strongly Strongly 
Post n_ disagree Disagree Agree agree 
a. | have established a clear vision for my school ns = mee ne 20 nat 
: Post 25 : 28% 52% 20% 
b. | have established an effective data management system to Pre 25 4% 48% 40% 8% 
ete performance data continuously throughout Post 25 AY, 36% 36% 2A% 
c. | ensure that our school's safety nets and interventions are Pre 25 4% 40% 48% 8% 
successful in getting struggling students back on track. Post 25 4% 20% 56% 20% 
d. | discuss why student depth of understanding is critical to Pre 25 16% 68% 16% - 
achieving our vision. Post 25 - 4% 76% 20% 
e. | search widely to benchmark our school’s performance and ‘Pre 25 12% 4A% AA% 2 
identify strategies that have enabled high-performing schools 
bi Post 25 - 32% 36% 32% 
to sustain improvement. 
f. | make judgments about teacher performance using a Pre 25 4% 32% 64% - 
variety of data sources, including student results. Post 25 - 4% 44% 52% 
g. In my school, | have institutionalized processes that ensure Pre 25 4% 60% 32% 4% 
that the professional development system is aligned with a 
belief that all students can achieve the same high standards. Post 25 7 28% 52% 20% 
h. | have strategies to use in order to buffer teachers from Pre 25 16% 52% 32% - 
distractions to their instruction. Post 25 4% 16% 56% 24% 


Note. Results include school-level respondents only. 
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Table E6. Twin Cities: Self-Reported Frequency of Interaction with Teachers and Use of Research 
Pre/ 


How often do you... Post n Never Seldom Sometimes Regularly 
= 0, 9 0, 
a. Discuss instructional issues with your teachers? ne cad zoe es le 
Post 25 - - 20% 80% 
' aes : Pre 25 16% 28% 44% 12% 
? 
b. Discuss equity issues with your teachers? Post 5 A% 0% 40% 52% 
Pre 25 4% 16% 60% 20% 
4a 5 : as 
c. Visit classrooms to briefly observe instruction: Post 25 i 36% 6A% 
d. Watch an entire lesson when observing classroom Pre 25 8% 20% 40% 32% 
instruction? Post 25 - 12% 36% 52% 
; : ; Pre 25 12% 52% 28% 8% 
? 
e. Model instructional strategies for teachers? Post 25 - 36% AA% 20% 
f. Give teachers specific ideas for how to improve their Pre 25 - 24% 60% 16% 
instruction? Post 25 - 4% 36% 60% 
g. Ask teachers about their use of data in instructional Pre 25 4% 32% 48% 16% 
decision-making? Post 25 - 12% 24% 64% 
: : : , Pre 25 4% 36% 40% 20% 
? 
h. Attend teacher professional learning community meetings? Post 25 - A% 18% 18% 
i. Provide or locate resources to help staff improve their Pre 25 - 28% 56% 16% 
teaching? Post 25 - 8% 36% 56% 
j. Made judgements about teacher performance based on Pre 25 4% 20% 72% 4% 
student-level data? Post 25 4% 4% 56% 36% 
k. Use research evidence in your own decision making? Ee 23 Be Bs bate , 
y 2 Post 25 - : 32% 68% 


Note. Results include school-level respondents only. 


Table E7. Twin Cities: Self-Reported Frequency of Community Engagement, Data Use, and Peer Observations 


Pre/ RY=\V(-) ¢-] | More than 

Post n Annually times/yr Monthly once a month 
a. You communicate to the community your Pre 25 64% 28% 4% 4% 
commitment to the achievement of high 
performance standards by all students? Post 25 36% nave 8% ao 
b. You rely on systematically collected evaluation Pre 24 50% 29% 21% - 
data about your school in decision making? Post 25 12% 40% 32% 16% 

: Pre 24 75% 21% - 4% 

c. Teachers observe each other's classrooms? bast 25 52% 36% 8% A% 


Note. Results include school-level respondents only. 


Table E8. Twin Cities: Perceptions of Teacher Practices 


Strongly Strongly 
disagree Disagree Agree agree 
a. Teachers in this school are able to accurately Pre 25 4% 44% 44% 8% 
monitor the progress of their students. Post 25 - 32% 52% 16% 
b. Teachers in this school have a sense of collective Pre 25 12% 32% 52% 4% 
responsibility for student learning. Post 25 4% 24% 40% 32% 
c. Teachers in this school have the knowledge and Pre 25 4% 28% 64% 4% 
skills they need to improve student learning. Post 25 - 20% 56% 24% 
d. Teachers in this school have ongoing conversations Pre 25 12% 44% 36% 8% 
among themselves about instructional practices. Post 25 - 24% 44% 32% 
e. Teachers in this school prepare all students to go Pre 25 - 32% 56% 12% 
on to some sort of post-secondary education. Post 25 4% 24% 48% 24% 
f. Teachers in this school view problems as issues to Pre 25 16% 48% 32% 4% 
be solved, not as barriers to action. Post 25 4% 40% 32% 24% 


Note. Results include school-level respondents only. 
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Table E9. Twin Cities: Perceptions of School-Level Practices 


Strongly 


a. We provide an aligned curriculum for students Pre 
across the grades. Post 
b. There is a strong commitment in this school to a Pre 
common set of shared goals. Post 
c. Our school improvement plan drives teachers’ Pre 
professional development. Post 
d. Administrators and teachers collectively plan who Pre 
will provide leadership for initiatives. Post 
e. The schedule allows for adequate embedded time Pre 
for collaborative teacher planning. Post 
f. We have a range of differentiated safety nets Pre 
available at the individual student level. Post 
g. Team leadership at all levels focuses on the Pre 
guidance, direction, and support of sustained 
improvement in instructional practice and student Bae 


learning. 
h. Most parents of students in this school expect their Pre 
children to go on to some sort of post-secondary 


education. Post 


25 
25 
25 
25 
25 
25 
25 
25 
25 
25 
25 
25 


25 


25 


25 
25 


Strongly 

disagree Disagree 

12% 36% 

8% 20% 

8% 40% 

- 44% 

4% 40% 

- 20% 

12% 28% 

4% 16% 

12% 28% 

8% 12% 

12% 56% 

- 48% 

8% 48% 

- 32% 

8% 8% 

4% 4% 


Agree 


44% 
48% 
40% 
24% 
44% 
56% 
48% 
36% 
44% 
52% 
28% 
40% 


36% 
40% 
52% 
44% 


agree 
8% 
24% 
12% 
32% 
12% 
24% 
12% 
44% 
16% 
28% 
4% 
12% 
8% 


28% 


32% 
48% 


Note. Results include school-level respondents only. 


Table E10. Twin Cities: Perceptions of District-Level Practices 
Pre/ 


Strongly 


Strongly 


a. District staff help to ensure that our schools offer an 
aligned curriculum. 

b. Instructional leadership is a key component of our 
principal evaluation system. 

c. District leaders often discuss instructional issues with 
school leadership teams. 

d. Our district offers a range of differentiated safety nets or 
interventions available at the individual student level. 

e. Our district has strategies to support principals and 
teachers in preparing students for college or other post- 
secondary education beyond high school. 

f. District administrators model high levels of professional 
practice. 

g. Our district culture is one in which all teachers and 
administrators feel responsible for working together to 
improve student achievement. 

h. Our district has institutionalized processes that ensure 
the PD program is aligned with a belief that all students 
can achieve the same high standards. 

i. Principals participate in ongoing high-quality professional 
development. 


j. Teaching is aligned to rigorous performance standards. 


k. Our district supports equity-related initiatives. 


Post 
Pre 
Post 
Pre 
Post 
Pre 
Post 
Pre 
Post 
Pre 


Post 


Pre 
Post 
Pre 


Post 
Pre 
Post 


Pre 
Post 
Pre 
Post 
Pre 
Post 


n 
26 
26 
25 
24 
26 
26 
26 
26 
26 


26 


26 
26 
26 


26 
26 
26 


26 
26 
26 
25 
26 
26 


disagree 


Disagree 
35% 
27% 
24% 
33% 
42% 
23% 
23% 
38% 
27% 
31% 

8% 
8% 
23% 
38% 
35% 
42% 
38% 
15% 
23% 
36% 
31% 
19% 


Agree 
58% 
50% 
68% 
25% 
46% 
38% 
58% 
50% 
62% 
58% 
77% 
54% 
50% 
19% 
46% 
31% 
46% 
31% 
54% 
40% 


42% 
50% 


agree 
8% 
23% 
8% 
42% 
8% 
23% 
8% 
8% 
8% 
12% 
15% 
31% 
15% 
38% 
8% 
15% 
4% 
31% 
15% 
20% 
15% 
15% 


Note. Results include both school-level and district-level respondents. 
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